Introduction

Research dissemination is a major step in the scientific process. As a first step to research translation, researchers and journal editors want the research they publish to reach the widest possible audience. The latter includes academics, clinicians, traditional media, the general public, and health policy services. A particularity of health-related science is that it can also be of interest to health consumers themselves as part of the shared decision-making process (Barry & Edgman-Levitan, 2012; Bredbenner & Simon, 2019; Carvalho et al., 2019; Stacey et al., 2017). This requires that any specific and technical scientific content from the original publication is also made available in an easy-to-understand format (Carvalho et al., 2019; Pushparajah et al., 2018). This has led researchers and journal editors to develop different tools to summarize research papers. For example, plain-language summaries use clear and to-the-point writing to improve communication and to reduce the time required to read and understand the research, providing readers with exactly what they need to know without using unnecessary words or expressions (Health NIo. Plain Language at NIH, 2022). In the field of physiotherapy, their number have doubled in the last few years, particularly due to their systematic use in reviews published by the Cochrane Library (Carvalho et al., 2019). Other types of summaries include audio abstracts (or podcasts) (Thoma et al., 2018), visual abstracts (Oska et al., 2020) and infographics (Thoma et al., 2018), which are visual representations of key features of a research aimed to rapidly provide information, directly useful for decision-making purposes (Otten et al., 2015). All of these, expressed in a fast-to-use and easy-to-understand format, can be disseminated easily and widely, for example through social networks (Brossard, 2013; Carvalho et al., 2019; Moorhead et al., 2013; Patthi et al., 2017; Peters et al., 2014), with the a priori aim to further increase the visibility of the research, and its use by health consumers, clinicians, health policies or future academic projects.

More recently, some video supports were suggested for medical communication such as massive open online communication (MOOC), blog, social media account, or video abstracts. The latter (including animations) have been proposed as an enjoyable way to disseminate research (Bredbenner & Simon, 2019; Spicer, 2014; Zong et al., 2019). Video abstracts usually length few minutes and explain, in a simple way, the background, the method, the findings of the study as well as the contribution to the field and/or take-home messages (Bredbenner & Simon, 2019; Spicer, 2014; Zong et al., 2019). According to the aim of the journal, videos can be scientific or educational but are generally freely available and will reach a wider audience if they are more accessible (Hub, 2022). This type of summary has been recognized as the most enjoyable compared with infographics, plain language summaries and usual abstracts (Bredbenner & Simon, 2019). Although video abstracts were found to be associated with greater citation counts in the field of physic sciences (Zong et al., 2019), a review conducted in 2013 only identified 10 scientific journals from the field of medicine that used video abstracts to promote their research reports and no previous study sought to explore the association between video abstracts and metrics of research papers in the field of medicine science (Spicer, 2014).

Citations, which are a widely available metrics for research papers, represent an opportunity to assess how visible was a given paper among the specific population of academic readers. Complementarily, electronic page-views of the online article and the Altmetric Attention Score [AAS; a multi-composite tool aimed to assess the social interaction of a research paper (Altmetric., 2020, 2021; Hook et al., 2018; Patthi et al., 2017; Trueger et al., 2015)] could also be used as tools to estimate the visibility of a given paper among the wider population of end users of the research (including academic readers but also more broadly clinicians, traditional media, the general public, health consumers and health policy services). Thus, both of these metrics could be used to address the question of whether video abstracts can improve the overall visibility of research papers.

Because the creation of this video abstract is time consuming, expensive and requires specific skills, it is important to assess whether it actually produces the effects it intends to, which has not been adequately investigated, particularly in the field of medical research.

Therefore, the aim of this study was to answer to following questions:

  1. 1.

    How video abstracts are associated with the citations of research papers?

  2. 2.

    How video abstracts are associated with the overall visibility of research papers among end users?

Method

Study design

We conducted a cross-sectional study of research reports published in the New England Journal of Medicine (NEJM) from 1 January 2018 to 31 December 2020. As such, ethical approval was not required. The NEJM was chosen because it is one of the leading journals of medicine research and has technical means ahead of the others. More recent reports were not considered (1) to allow maximal time for articles to be disseminated, viewed, and cited (Vaghjiani et al., 2021) and (2) limit the influence of freely available or subscriber only access. They were identified from the issue index of the journal (https://www.nejm.org/medical-index?query=main_nav_lg) and the type of promotion was recorded (video abstract or no video abstract). Video abstract from the NEJM, called “Quick Take” usually length 2 to 3 min. According to the NEJM website, “Quick Take videos present highlights of important articles in an engaging, animated format. These brief videos offer an innovative way to understand important new research findings that have an impact on medical practice and patient care”. Quick Take are directly available on the article web page on the NEJM website and are also hosted on Youtube website (www.youtube.com) and disseminated through to Twitter (twitter.com/home).

Additionnaly, research reports were categorized according to the design of study (RCT, including cluster RCT and non-RCT) and the medical specialty as displayed on the NEJM website (allergy immunology, cardiology, clinical medicine, dermatology, emergency medicine, endocrinology, gastroenterology, genetics, geriatrics/aging, hematology/oncology, infectious disease, nephrology, neurology/neurosurgery, obstetrics/gynecology, pediatrics, psychiatry, pulmonary/critical care, rheumatology and surgery). The number of pages, the first author’s country affiliation (Australia, Canada, UK, USA, Europe and others) as well as the presence or not of funding were retrieved as potential confounding factors. The number of characters in the title or in the abstract, the number of authors and the number of references were not recorded due to their limited contribution in models aimed to assess the effect of visual or video abstracts on Altmetric score or citations (Aggarwal, 2021; Zong et al., 2019). The data were collected between March 3 and April 2, 2022.

Article selection criteria

Research reports published in the original articles section of the NEJM were screened for eligibility. Research reports were not eligible if they were retracted, were published in short report format (because none benefitted from a video abstract) or had benefited from a different type of promotion (such as infographic or interactive infographic). Those related to COVID-19 were only included in a sensitivity analysis.

Primary outcome

The primary outcome was the number of citations. Total citations for each paper were extracted from the Dimensions database (https://www.dimensions.ai/) (Hook et al., 2018).

Secondary outcomes

Views of the research reports were collected directly on the NEJM website. The AAS for a research output provides an indicator of the amount of attention that it has received (Altmetric, 2021; Trueger et al., 2015). It is derived from an automated algorithm, and represents a weighted count of the amount of attention research output received from social interaction (Altmetric, 2021). For example, this includes news, blog, policy documents, patent, Wikipedia, peer review, number of Mendely readers, number of dimensions in the Web of Science citations, Youtube, and social networks such as Twitter (tweets and retweets) and Facebook. AAS was extracted for each research report from the Dimensions database (Hook et al., 2018).

Statistical analysis

The normality of the data was assessed using the Kolmogorov–Smirnov test. Categorical data were expressed as counts (%) and continuous data were expressed as mean (SD) or median (25th–75th percentile) according to the distribution.

Due to the skewness of data, an inverse binomial regression was used to assess factors influencing citations, views and AAS. The model included the presence of video abstracts as well as other independent covariables (study design, first author’s country, page number, funding, field) as potential confounding factors (Aggarwal, 2021; Zong et al., 2019). Medical fields with less than 10 research reports were pooled within an “others” category. Additionally, the time frame between the date of publication (corresponding to the date of the issue in which the paper was published in) and the date on which the metrics were collected was also included in the model. Model fitness was compared with a Poisson regression using the likelihood ratio chi-square statistic.

Finally, in order to better interpret the relevance of the findings, results were expressed as incidence rate ratio (95% CI). A change of at least 10% in the incidence rate ratio of citations, views or in the AAS was a priori considered worthwhile. A sensitive analysis including research reports related to the COVID-19 was conducted to explore the extent of the attention related to this topic and the replicability of the initial findings in this context of strong media coverage.

Graphpad Prism 8.4.2, MASS and foreign packages for R software were used for analysis.

Results

Studies

A total of 640 studies were screened for eligibility and 500 were included (Fig. 1). Characteristics of the included studies are shown in Table 1. Briefly, the median time from publication was 3.0 (2.2 to 3.6) years, 72% were RCTs, 49% were from the USA and the most represented field was hematology/oncology (26%).

Fig. 1
figure 1

Study flow

Full size image
Table 1 Characteristics of the included research reports
Full size table

Primary outcome

The presence of video abstracts was associated with an increase in citations (IRR 1.15), although this estimate came with uncertainty ranging from virtually no effect to a worthwhile effect (95% CI 0.98 to 1.35, Fig. 2A). Further details are available in Table 2.

Fig. 2
figure 2

Incidence rate ratio of the association between video abstracts and metrics of research reports from a negative binomial regression model (A) and incidence rate ratio of the association between video abstracts and metrics of research reports from a negative binomial regression model, including research reports related to the COVID-19 (B). IRR incidence rate ratio

Full size image
Table 2 Incidence rate ratio (95% CI) from the negative binomial regression model
Full size table

Secondary outcomes

Video abstract was associated with a worthwhile increase in views (IRR 1.35, 95% CI 1.18 to 1.54) but this estimate came with uncertainty ranging from a just worthwhile benefit to a very worthwhile benefit (Fig. 2A). It was also associated with a worthwhile increase in AAS (IRR 1.25) but this estimate also came with uncertainty, ranging from a trivial benefit to a very worthwhile benefit (95% CI 1.08 to 1.44, Fig. 2A). Further details are available in Table 2.

Sensitivity analysis

Twenty-six research report related to the COVID-19 met the inclusion criteria. Their characteristics are available in Table 3. COVID-19 was associated with a very worthwhile increase in the citations (IRR 23.20, 95% CI 15.50 to 35.21), views (IRR 31.69, 95% CI 22.97 to 44.14) and AAS (IRR 17.86, 95% CI 12.36 to 26.11). However, this did not change the association between video abstracts and citations, views and AAS (Fig. 2B and Table 4).

Table 3 Characteristics of the research reports related to the COVID-19
Full size table
Table 4 Incidence rate ratio (95% CI) from the negative binomial regression model including research reports related to the COVID-19
Full size table

Discussion

The main findings of this study were that video abstracts were associated with a worthwhile increase in views. They were also associated with an improvement in citations and social attention, although the benefit may be small. Additionally, the COVID-19 largely fostered attention in increasing citations, views and social attention. However, even in this context, video abstracts remained an independent predictor of citations, views and social attention.

A cornerstone of the academic model is scientific publication. Citation counts is therefore essential for researchers to increase the recognition of their expertise in a given field, their visibility and that of their institutions as well. It is also crucial for journal editors because of the way the impact factor is calculated (Patthi et al., 2017). Although largely discussed (Callaway, 2016), this metric is associated with research quality (Lee et al., 2002) and is also used to rank journals. A good ranking is therefore essential for journals to attract more quality research. Interestingly, our results extent those from Zong et al. who also found in the field of physics science, that video abstracts were able to increase citations counts by approximatively 20% (Zong et al., 2019). Waiting that future studies further clarify the effect size of the association between video abstracts and citation counts, the present results should already encourage researchers and journal editors to work together to produce suitable video abstracts in order to improve the academic visibility of research reports.

Accordingly, the production of a video abstracts is intrinsically related to its objective (scientific or educational) and therefore to the targeted audience (academics or a more general public, respectively). Although the case studied (“Quick Take” from the NEJM) was mostly scientific videos, we found that they were also able to improve views and social interaction (although the benefit may be small for the latter), suggesting a larger dissemination through to the general public, including health consumers. The dissemination of robust medical scientific data to patients is particularly important because many of them frequently seek for medical information through to the internet (Pushparajah et al., 2018). Helping them to be more knowledgeable about their medical condition is therefore an important step in the shared decision-making process (Barry & Edgman-Levitan, 2012; Pushparajah et al., 2018; Stacey et al., 2017). However, effectively disseminating research results through to social networks is only the first step of research translation, particularly for lay public. It also implies that the information is correctly understood and requires the communication support to be adapted to lay people. For example, it has been shown that only 91 out of 4 421 plain language summaries (2%) from the physiotherapy field were written at a suitable reading level for a lay person (Carvalho et al., 2019). Fortunately, previous research found that video abstracts were equally or more understandable and enjoyable than plain language summaries (Bredbenner & Simon, 2019) but further studies are still required to assess the level of information retention among healthcare consumers and how it may improve the shared decision-making process. Particularly, whether educational designed video abstracts would have further improved views, social interaction and information retention compared with scientific video abstracts such as in the case studied, would require further investigations.

A limitation to large scale production of video abstracts is their cost, which has never been addressed in cost-effectiveness analysis yet. Although the British Medical Journal Author Hub states that they should not be expensive to produce by the authors (Hub, 2022), it requires specific equipment and/or technologies, as well as time. When produced by the journal itself, it involves multimedia and editorial teams and/or a production company, which is also costly. In this context, the effect of video abstracts should be maximized by disseminating them through to the website of the journal but also through to many other medias such as video platforms, professional research translation websites, concerned patient association websites and social networks (Vaghjiani et al., 2021). However, the optimal journal self-promotion strategy is unknown yet and future studies should also focus on this topic.

Finally, we were concerned that the scientific and mediatic attention related to the COVID-19 may have drown out the effects of video abstracts. The present findings confirmed that COVID-19 fostered many attentions. For example, compared with cardiology and hematology/oncology, medical fields usually associated with highest rate of citation (Kulkarni et al., 2007), COVID-19 conservatively had four time as many citations, seven time as many views and five time as many social attention. However, we found similar association between video abstracts and citations, views and AAS when including COVID-19 related research reports in the model, suggesting that our findings were replicable in the COVID-19 area (ie. COVID-19 research reports that benefited from a video abstract were also more cited, viewed and had more social attention).

Strength and limitation

Some strength of this study includes it large sample size, studies matched for publication date (each issue of the NEJM is released with at least one Quick Time), a multivariate analysis including identified confounders as well as a sensitive analysis to explore the consistency of the results in the highly publicized environment of the COVID-19.

An important limitation was that the choice to promote an article with a video abstract was not randomized and may have led to a selection bias from the NEJM editorial board by choosing the supposedly most popular research reports. For example, RCT design may increase citation counts, views and AAS [(Aggarwal, 2021) and Table 2] and were highly represented in the video abstract group of the present study. Additionally, we only included research reports from a single, extremely visible, journal. This limits the generalizability of the findings for less visible and more specialized journals. However, this also limited the potential bias of heterogeneity and variation in formatting, publication frequency, and differences in self-promotion among multiple journals which may have influenced on the outcomes. For example, the NEJM uses self-made video abstracts while most journals request the authors to perform their own video abstracts (Spicer, 2014; Zong et al., 2019). Even though guideline helps authors in building their video abstracts (Spicer, 2014), this raise question about heterogeneity in the content and the quality of the record. Another limitation is that the number of views was retrieved directly from the NEJM website which may have underestimated the true estimation because it does not capture all the potential view sources, such as those from PubMed. Finally, we found that video abstracts were associated with citations, views and AAS but causation cannot be confirmed due to the nature of this study. According to these limitations, we suggest future randomized controlled trial to address how cost-effective are video abstracts in improving the visibility of research papers compared with no video abstracts, across multiples journals and using a standardized format of video abstract and journal self-promotion strategy.

Conclusion

Video abstracts as a tool to disseminate health science through to social networks are associated with a worthwhile increase in the number of views of research reports. They are also associated with an increase in citations and social attention, although the benefit may be small. These results were consistent in the highly publicized environment of the COVID-19. Future studies should be performed to refine these estimates, assess their replicability across a wider range of journals and to estimate the cost-effectiveness of video abstracts. Finally, the relative effect of different types of video abstracts (scientific or educational) as well as the relative effect of video abstracts compared with other communication tools, such as plain language summaries, podcast or infographic should also be explored.