Introduction

The ongoing pandemic has disrupted the clinical practice and reduced direct patient interaction in all specialties of medicine and surgery, including ophthalmology [1]. This has severely impacted the surgical training for ophthalmology residents and glaucoma fellows during the past two years [2]. Trabeculectomy and glaucoma drainage devices (GDDs) implantation are the two most performed surgical procedures for glaucoma management. Among the GDDs implantation, the most widely taught technique to the trainees is the implantation of the valved GDD, Ahmed Glaucoma Valve® (AGV, New World Medical Inc., Rancho Cucamonga, CA), as it is the primary procedure for managing glaucoma, especially in eyes with a high risk of trabeculectomy failure [3].

Although simulation-based surgical education is an excellent alternative for trainees until they attain the requisite levels of competence to perform surgery on patients, however not many academic institutions have access to these state-of-the-art simulators [4]. Therefore, the trainees resort to developing foundational skills for surgical procedures through online videos [5]. The easy access to didactic videos on surgical techniques and complication management has become an indispensable educational tool for many practicing ophthalmologists worldwide. Moreover, the evidence suggests that surgical “tips and tricks” are better grasped through video resources than by reading handbooks or scientific peer-reviewed articles [6, 7]. Along with YouTube, the largest open-access video sharing platform, several websites are dedicated to showcasing surgical videos of various ophthalmic subspecialties [5]. Although these platforms have become a widely accessed medium for sharing educational content, however, lack quality control due to the absence of a standardized peer review process, leading to concern over the accuracy of instruction [8]. In this study, we objectively evaluate the quality of videos on AGV implantation to guide surgical trainees in choosing the best resources for gaining foundational knowledge about AGV valve implantation.

Methods

This study did not involve participation by any live human or animal subjects. Since all the data used for analysis was publicly available, de-identified, and anonymized, an institutional review board approval was not required.

Study design and search strategy

In this observational study, 23 websites with surgical training videos (Table 1) were independently queried by a glaucoma specialist (PI) on May 27, 2022, using the search keyword “Ahmed glaucoma valve implantation.” All the websites included in the study were freely accessible and had ophthalmology content. No changes were made to the standard search preferences on these websites, and the search queries were conducted without any user login except VuMedi (www.vumedi.com), which requires registration to search videos. The first 100 video search results from YouTube were included as they covered the first five pages of the search (20 videos/page × 5 = 100 videos), assuming most users would not go beyond the first five pages of the search results. On the other websites, all video search results were included in the analysis. In addition, the URLs were noted for further evaluation and quality assessment.

Table 1 Websites searched for video-content related to Ahmed glaucoma valve implantation
Full size table

Inclusion and exclusion criteria

From the search results, all surgical videos in English were included in the study. However, the videos about patient surgical experiences (n = 4), physician experience (n = 2), revision surgeries (n = 4), complications associated with valve implant (n = 3), surgery performances on animal eyes (n = 3), combined surgeries (n = 11), videos describing priming and functioning of AGV (n = 4) and surgical implant videos using valves other than the AGV (Baerveldt glaucoma implant, Aurolab aqueous drainage implant)(n = 4) were excluded from the analysis.

Content characteristics

The view count, time since upload, number of likes, comments, and video length were recorded. In addition, the origin of the video (North & South America, Europe, Asia, Africa) and the presence of audio narration and descriptive subtitles indicated on the website were also recorded. The uploaders were ophthalmologists, academic institutions, optometrists, commercial entities, and patient support groups.

Quality assessment

To objectively evaluate the quality of content in the videos, each video was scored using the three well-established, standardized scales—Sandvik score, Health on the Net Foundation Code of Conduct (HON code), mDISCERN scoring system, and the Global Quality Score (GQS). The Sandvik score ranges from 0 to 14 points, where 14 represents optimal quality, 11 to 14 represents excellent, 6 to 10 represents medium, and 0 to 5 represents poor quality. The guidelines for the Sandvik scoring system are detailed in Supplementary Table 1. The HON code is specifically designed for quality assessment of medical- and health-related information on the internet. HON code consists of eight principles for evaluating the reliability and credibility of health information. The videos were scored either 1 (adherent) or 0 (nonadherent) for each of the eight principles. The scoring guidelines for the HON code scoring system are detailed in Supplementary Table 2.

The GQS, described by Bernard et al., is based on a 5-point Likert scale and considers the flow, ease of use, and video quality of the information presented in the online videos. The score ranges from 1 (poor quality) to 5 (excellent quality). The detailed guidelines for GQS are outlined in Supplementary Table 3. The mDISCERN scoring system evaluates the clarity, reliability, bias, presentation of additional information sources, and content uncertainty. The maximum score is five points, one for each criterion (Supplementary table 4). Higher scores indicate higher reliability of videos.

For quality assessment, 14 steps of AGV implantation surgery were taken into consideration; these steps included surgical time-out, eye draping, and placement of speculum, exposure of the surgical site, conjunctival peritomy, hemostasis, priming the AGV, anchoring the plate, creation of scleral track (if performed), tube track and entry location, trimming of the tube, tube insertion, tube fixation with 10-0 nylon suture, scleral/corneal patch graft (if used) and lastly, conjunctival closure [9]. The video quality score (VQS) was determined for each video based on these steps. The scores were divided into the 0 to 4, 5 to 7, 8 to 10, 11 to 12, and 13 to 14-point ranges, and the ranges were rated as having very poor, poor, fair, good, and excellent quality, respectively.

Two different authors (PI and RBS) evaluated all videos independently on two different computers in two different countries (India and the USA). Additionally, the cache memory of the browsers was cleared to prevent the impact of previous exposure in the appearance of results on the first five pages. Before the statistical analysis, the intraclass correlation coefficient (ICC) was performed to assess the consistency of the scores given by both ophthalmologists. The correlation coefficient was obtained at greater than 0.9 for all scoring systems. A mean of the scores given by the two ophthalmologists was used for statistical analysis. A few videos were rechecked randomly by all four observers (PI, RBS, UPSP, and RC) to ensure the accuracy of the collected data. The content was categorized based on the uploader type, video origin, presence/absence of descriptive subtitles, and the presence/absence of audio narration, and the respective quality assessment scores were analyzed.

Statistical analysis

The data were analyzed using R Studio version 4.1.2 (R Foundation for Statistical Computing, Vienna, Austria). The values were stated at a 95% confidence interval (CI). In addition, the descriptive statistics were recorded as percentage, mean ± standard deviation (SD), median, and range.

An independent t test and a one-way ANOVA test were used to analyze continuous data. The Spearman correlation analysis was performed to correlate the scores of the quality assessment parameters. The linear regression analysis was done to correlate video parameters and quality. All p values < 0.05 were considered statistically significant.

Results

This study included videos from the most widely used medical education resources (n = 23) to evaluate the instructional quality of videos for AGV implantation. The website URLs and search results from these websites are enlisted in Table 1. A total of 119 videos were included in the initial evaluation, and 35 were excluded from the final analysis. The descriptive parameters of the remaining 84 videos are outlined in Tables 2 and 3.

Table 2 Summary of parameters analyzed for the AGV implantation videos
Full size table
Table 3 Video parameters analyzed for the AGV implantation videos
Full size table

All the videos showed tube insertion in the scleral tract, and 97.6% (82/84) videos showed scleral plate anchoring at an adequate position. The three steps that were most commonly variable in the videos were—the creation of a scleral track [a partial thickness rectangular scleral flap of adequate depth with an additional trench fashioned over the scleral bed for the tube insertion, 32.14% (27/84)], tube fixation with a 10-0 nylon suture (60.7%, 51/84), and hemostasis with cautery efficiently and precisely (70.23%, 59/84). The mean Sandvik, HON Code, GQS, mDISCERN, and VQS scores for all the videos were 11.79 ± 1.70, 6.86 ± 0.75, 3.97 ± 0.93, 3.26 ± 0.66, and 11.45 ± 2.67, respectively. The mean Sandvik, HON Code, GQS, DISCERN, and VQS scores for ophthalmologists and stand-alone eye clinics were 11.58 ± 1.76, 6.86 ± 0.8, 4.09 ± 0.34, 3.22 ± 0.66, and 11.49 ± 2.67, respectively. The mean Sandvik, HON Code, GQS, DISCERN, and VQS scores for academic institutions were 12.16 ± 1.74, 6.66 ± 0.77, 3.83 ± 0.57, 3.58 ± 0.9, and 11.25 ± 3.01, respectively. We observed a statistically significant difference in GQS scores between the countries of origin (p = 0.02). Only a third (34.5%; 29/84) of the videos were of excellent quality; nearly half, 48.81% (41/84) were of good quality, 10.71% and 5.95% were of fair and poor quality, respectively. The scores based on the uploader type and origin are summarized in Table 4.

Table 4 Different scores for different uploaders and origin continents
Full size table

The video duration was significantly associated with a higher GQS score (p = 0.023), whereas the comment number was significantly correlated with a higher Sandvik score (p = 0.042). The statistical analysis of the correlations is presented in Table 5. The educational quality of videos with the narration was significantly higher in the mDISCERN, GQS, and HON code scoring system than in videos without narration. The quality of videos with descriptive subtitles was significantly higher in VQS and HON code scoring systems compared to videos without descriptive video subtitles (Table 6). The consistency was excellent between the two ophthalmologists regarding the four scoring systems (ICC, respectively: 0.92, p < 0.001; 0.93, p < 0.001; 0.94, p < 0.001 and 0.90, p < 0.001).

Table 5 Correlation of video parameters with the scoring systems
Full size table
Table 6 Comparison of videos with and without video narration or descriptive subtitles
Full size table

Discussion

Glaucoma is one of the leading causes of irreversible visual impairment globally. Surgical interventions are needed in patients who are refractory to medical treatment. In the past few years, there has been a steady shift in practice patterns with an increase in GDD implantations and a decrease in filtering procedures [10]. Among the available GDDs, AGV offers the most favorable risk and efficacy profile. Therefore, surgical training for AGV implantation has been widely adopted in the surgical training programs for ophthalmology residents and glaucoma fellows.

The digital resources, especially educational videos, are becoming a widely used resource by the training programs. Although these digital resources cannot substitute clinical training, lockdowns in response to the COVID-19 pandemic have further expanded the utilization of these readily available resources. YouTube and other video-sharing platforms are frequently used for physician and patient education [5]. We accessed the content on 23 ophthalmology video hosting websites and found the AGV implantation technique only on five websites with the highest number of videos available on YouTube. The information from YouTube must be interpreted with extra caution as it is expected that the popular YouTube channels that already have a lot of subscribers and videos will have a greater number of views and “likes” when compared to videos of the same quality from channels with fewer number of subscribers. It is important to consider that the video indication algorithms used by platforms such as YouTube influence the number of views of each video. The lack of content of the most performed glaucoma procedure could not be found, thus revealing the inadequacy of these educational platforms. Most uploads (67.8%) were from ophthalmologists in non-academic practices, followed by 14% from academic institutions.

The objective quality assessment of all videos according to Sandvik, HON Code, GQS, DISCERN, and VQS scores was 11.79 ± 1.70 (excellent quality), 6.86 ± 0.75 (excellent quality), 3.97 ± 0.93 (good quality), 3.26 ± 0.66 (fair quality), and 11.45 ± 2.67(good quality), respectively. This indicates that the quality of surgical videos was significantly variable. Additionally, the algorithm-generated bias on YouTube and lack of peer review is a cause of concern regarding the reliability of content posted on these websites. Some previous studies evaluated the educational surgical video quality ranging from good to poor, as indicated by Sandvik, HON Code, GQS, DISCERN, JAMA, or VQS scoring systems [11, 12]. This is the first study to assess the education content quality for AGV implantation videos. The video content ranks primarily for the title characters; therefore, for better outcome, more content-related keywords must be included in the transcript. Out of the 14 steps, only 12 were highlighted in most videos. The creation of a scleral flap was the most variable and least emphasized step in the procedures. All the videos failed to address the conditions for adopting a scleral flap or scleral tunnel and the complications and benefits of each. In the videos, the specifications of AGV were mentioned only in 18 videos; 11 had FP7, five had PC7, and two used FP8. Among the five most viewed surgical videos, ophthalmologists in non-academic practice uploaded four, and the remaining one was by a channel run by an international teaching hospital (Wills Eye Hospital, Philadelphia, PA). Expectedly, the videos with the most views also had the highest number of likes, while the most number of comments was on the video with the second-highest view count. Interestingly, higher VQS scores were observed among the videos uploaded by commercial entities; however, these videos only formed a small percentage of the total videos (a YouTube channel by “D.O.R.C [Dutch Ophthalmic Research Center] International” contributed 37.5% [3/8 videos]).

The videos were primarily uploaded (39%) from sources in the USA, followed by Asia (22%) and Europe (13%). Among the analyzed videos, 57% of videos had audio narration pertaining to surgical procedure steps, while the remaining 42% had no sound or background music. Furthermore, only 27% of videos included descriptive subtitles in the videos. The surgical video quality was significantly better in videos with narration or descriptive subtitles than in silent videos.

Correlation between various assessment scores are listed in Table 7. We observed a correlation between video quality and duration and a moderate correlation with daily comments count. There was no significant correlation with other descriptors such as total and daily views, total and daily likes, and total comments. The video quality assessment suggests that the view count and the number of likes, and comments are unreliable indicators of video quality, which is in harmony with our analysis [13]. A recent systematic review with pooled data from 11 studies has shown that basic surgical skills can be taught as effectively through online video-based education as conventional teaching methods [14]. Another review of 22 studies has shown that video-based surgical education is effective for different levels of surgical training; however, good study quality and follow-up are needed to know the most impactful aspects of video-based interventions [15].

The limitation of our study is the dynamic nature of these platforms, causing a change in the view counts of these videos. The ever-increasing role of the internet in our daily lives has led to a growing reliance on video platforms for the patient and student education; therefore, it is essential to adapt to the newer teaching practices. However, the need of the hour is to develop consensus guidelines to guide the sources and platforms to perform peer review to standardize these videos.

Table 7 Correlation between various assessment scores
Full size table