We tested the following video players: AblePlayer; Acorn; Adobe; AFB; Amazon; AMI Player; Brightcove; Facebook; JW Player; Kaltura; MediaElement; MediaSite; Ooyala; OzPlayer; Panopto; PayPal; Plyr; RAMP; Video.js; Vidyard; Vimeo; Viostream; Wistia; Yahoo; YouTube; and YouTube embed.
We conducted four rounds of testing:
-
Round 1: Desktop testing on Google Chrome, Windows 10;
-
Round 2: User testing with a vision-impaired user using a screen reader;
-
Round 3: Mobile testing on iPhone and Android devices; and
-
Round 4: Mobile testing on an iPad with a Bluetooth keyboard.
We identified certain “show-stoppers” that were failures of the four non-interference clauses in WCAG2: If technologies are used in a way that is not accessibility supported, or if they are used in a non-conforming way, then they do not block the ability of users to access the rest of the page [13].
3.1 Round 1 Testing
Initially we conducted testing on Google Chrome version 61.0.3163 under Windows 10. A series of tests were undertaken including whether the video player supported audio descriptions, whether the transcript was accessible to the keyboard and whether the volume of the player could be modified. Video players were deemed to include show-stoppers if any of the following occurred:
-
Audio plays automatically; unless the user is made aware this is happening or a pause or stop button is provided – failure of WCAG2 Level A Success Criterion 1.4.2: Audio Control);
-
Video contains a keyboard trap (i.e. users cannot escape from the video using the keyboard) – failure of WCAG2 Level A Success Criterion 2.1.2 No Keyboard Trap; and/or
-
Full-screen video contains a reverse keyboard trap (i.e. users cannot escape from full-screen mode using the keyboard) – failure of WCAG2 Level A Success Criterion 2.1.2 No Keyboard Trap.
At the conclusion of Round 1 testing only the following eight video players remained: AblePlayer; JW Player; Kaltura; OzPlayer; Panapto; Plyr; and YouTube embed.
3.2 Round 2 Testing
Experienced vision-impaired users tested the remaining eight video players on the following combinations of screen reader/operating system and browser:
-
JAWS with Windows 10 with: Internet Explorer; FireFox; and Chrome
-
NVDA with Windows 10 with: Internet Explorer; FireFox; Chrome; and Edge
-
VoiceOver with iOS Safari
-
TalkBack with Android Chrome
A series of tests were undertaken including whether the player’s controls were labelled, button status was announced appropriately and fast-forwarding and rewinding was available to the screen reader user. A video player was deemed to include a show-stopper if the video could not be played– not a failure of WCAG2, but deemed a significant failure. Only one video was excluded at the end of this round of testing: Panopto.
3.3 Round 3 Testing
We tested the remaining seven video players Google Pixel 1, Android 8.0, Chrome and iPhone 7+, iOS 10.3.2, Safari. A series of tests were undertaken including whether the mobile video player supported captions and/or audio descriptions, whether the volume of the video player could be modified and whether color contrast was sufficient. A video player was deemed to include a showstopper if any of the following occurred:
-
Video could not be played – not a failure of WCAG2, but deemed a significant failure;
-
Video could not be paused) – failure of WCAG2 Level A Success Criterion 2.2.2 Pause, Stop, Hide; and/or
-
Video crashed the browser – not a failure of WCAG2, but deemed a significant failure.
Three video players: Kaltura, MediaSite and YouTube contained embedded show-stoppers. The remaining four video players were: AblePlayer; JW Player; OzPlayer; and Plyr.
3.4 Round 4 Testing
We tested the remaining four video players on an iPad, iOS 10, Safari, Zagg keyboard. We tested only one task via the keyboard on the iPad: whether the controls could be operated by the keyboard. A video player was deemed to include a showstopper if any of the following occurred:
-
Video could not be played – not a failure of WCAG2, but deemed a significant failure;
-
Video could not be paused – failure of WCAG2 Level A Success Criterion 2.2.2 Pause, Stop, Hide; and/or
-
Video crashed the browser – not a failure of WCAG2, but deemed a significant failure.
Two video players: JW Player and Plyr, contained show-stoppers. The remaining two video players were: AblePlayer; and OzPlayer.