This work presents design challenges we have encountered and our implemented solutions – focusing on scenarios that highlight design to support visual and non-visual access. For each challenge we summarize the specific goals of a desired interaction sequence, the challenges we sought to address, and our implemented solution. All challenges and solutions take into consideration interviews with users with and without visual impairments, discussions with expert screen reader users, as well as the general expertise of PhET’s design team and pedagogical expertise of PhET’s science content and pedagogy experts. The result is that some design decisions and aspects of the resulting solutions stem from unique pedagogical needs of the sims and may not be a direct result from interpreted or stated user needs or preferences from interviews.
3.1 Custom Slider Features to Support Content Learning
The sim Resistance in a Wire (Fig. 2) is used in high school and introductory college physics courses to teach the relationship between the resistance of a wire and its resistivity, length, and area. The sim consists of an equation (R = ρL/A) and a piece of a wire, with three sliders with various input ranges to adjust the values of resistivity (ρ), length (L), and area (A). Incrementing or decrementing the slider range values results in a change to the size of the variables in the equation and parameters of the piece of wire. For example, increasing the length slider value results in an increase in size of the variable L in the equation and a corresponding increase in size of the variable R, while simultaneously increasing the length of the piece of wire. Decrementing the resistivity slider value results in a decrease in the size of the variables ρ and R in the equation, and decreasing the number of small black dots in the piece of wire, which represent impurities.
The desired interaction sequence with this sim is for students to initially explore by changing the sliders’ input values with a qualitative focus, changing the input values to extremes (e.g., the maximum, minimum, and median values) and observing how the variables and the piece of wire change. Then, as students continue manipulating the sliders, to compare changes across the mathematical and physical representations, building connections between the mathematical equation and a real-world example (the physical piece of wire). Another useful interaction sequence would be for students to make quantitative comparisons, using the numerical slider values to set up comparisons convenient for simple mental calculations – typically doubling or halving values. For example, comparing two scenarios with the same resistivity, but one scenario having double the length and area as the other scenario (both scenarios result in the same resistance). Attending to the quantitative slider values can also be utilized by teachers in the classroom to direct students to specific scenarios for exploration, calculations, or discussions.
When implementing alternative input capabilities and descriptions, we wanted to ensure we provided access to the sliders, and that this included intuitive and efficient access to: (1) the extreme values of each input range, with description highlighting the extreme size differences in the equation variables that result, (2) “double” and “halve” slider values, and (3) all values in the input range.
All interactive objects in the sim were instrumented to support alternative input access and descriptions were designed and implemented to support non-visual access. Here we focus on the navigation and description capabilities associated with the sliders.
We initially implemented each slider with the standard and recommended native HTML slider attributes to support access to the input type range. Users could press Tab, or other common navigation approaches, to move focus to each of the three sliders. Once a slider had focus, Arrow keys could be used to increment (Up/Right Arrow keys), or decrement (Down/Left Arrow keys) the input values. The initial step size for each slider was 1/100th of the slider range. Using the Home/End keys resulted in jumping to the maximum/minimum input range. Using the Pg Up/Pg Down keys resulted in incrementing/decrementing the input value by 1/10th of the slider range. As the slider is adjusted by the student, a description of the resulting slider input value is provided, followed by an alert describing the relative size of the equation variables – e.g., “As letter ρ grows, letter R grows. Resistance is now 0.82 Ohms.”
The initial implementation effectively supported efficient access to the maximum and minimum slider values (through the use of Home/End, or alternatively, multiple presses of the Pg Up/Pg Down keys), but did not provide efficient access to easily doubled or halved input values, or to all possible input values. These issues are a result of the extreme differences in the input range for the ρ (resistivity) slider as compared to the length and area sliders (see Table 1). To support effective qualitative comparisons, the range must span values that students perceive as physically small to large (e.g., length ranging from 0.10 cm to 20.00 cm). Minimum values must be non-zero to avoid zero or undefined resistance.
In the initial implementation, using the arrow keys to adjust the input values would result in default step sizes of 0.0099 for ρ, 0.199 for Length, and 0.149 for Area, as typical HTML range inputs require the range to be evenly divisible by the step size for the whole range of values to be accessible. These are not intuitive numbers for comparisons, or mental mathematical manipulation like double or halving. Figure 3 compares the slider panel in Resistance in a Wire with quantitative values not rounded (Fig. 3, left panel), and rounded (Fig. 3, right panel). Such a small default step size could also have the undesirable outcome of initial changes to the input value resulting in very small changes to the equation and piece of wire, obscuring important qualitative comparisons readily apparent with larger changes to the input values.
To address these challenges, we first implemented the sliders with the standard native HTML features, and then added two additional new features: a modifier key and a custom variable step size. The new modifier key is the Shift key. Used in combination with the Arrow keys, i.e., Shift + Arrow key, increments/decrements the input value by the smallest allowable step size. For this sim, across all sliders, this is 0.01. Using standard ways of adjusting the input value of a slider – Arrow keys, Home/End, Pg Up/Pg Down – allow the user to efficiently span the range of input values, and use of the Shift + Arrow keys allows fine control when targeting a specific input value. We will implement this modifier key for all sliders in PhET sims, for consistency.
A variable step size was also implemented, to support more efficient access to input values that support simple mental calculations. There is a base step size used through most of the input range. This base step size is varied to address two specific scenarios – ensuring the minimum and maximum values of the sliders can always be reached, and ensuring that rounded input values (easier for mental mathematical calculations) are reached when transitioning between different slider navigation approaches. For the ρ slider, with a range of 0.01–1.00, the base step size is used across the range 0.05–1.00. When decrementing with an Arrow key from an input value of 0.05, the input value changes to 0.01 the minimum input value (a step size of 0.04) – ensuring the Arrow keys support navigation to the extreme ends of the input range. When using the Pg Up/Pg Down keys, or the Shift modifier key to access an input value not an increment of 0.05, incrementing/decrementing with an Arrow key will result in an input value that is the nearest increment of 0.05. Similarly, the Length and Area sliders have a base step size of 1.00. When decrementing by Arrow key from an input value of 1.00, the input value changes to the minimum value – 0.10 (Length) and 0.01 (Area). When an input value is accessed that is not an increment of 1.00, incrementing/decrementing by Arrow keys results in the nearest input value that is an increment of 1.00.
The result of this implementation are sliders that can be accessed in all of the typical ways – meeting expectations of interaction with sliders for users with and without visual impairments – and that address the pedagogical goals of supporting access to (1) the extreme values of each input range, (2) values easy to “double” and “halve”, and also (3) all values in the input range.
3.2 Interacting with Uncommon Objects Using Native HTML Elements
The sim John Travoltage (Fig. 4) is used from elementary school through introductory college level to introduce the topic of static electricity. The sim opens with the character John, standing in a room, on a rug, next to a door. He has a moveable leg and arm. Rubbing his foot on the rug results in electrons, represented as blue circles with a “−” sign on them, transferring from the rug onto John’s body. John’s arm can be moved closer, or farther, from the nearby doorknob. Moving John’s arm closer to the doorknob can result in the electrons on John’s body discharging into the doorknob, resulting in a shock. A productive exploration of this sim typically includes students exploring the relationship between the amount of charge on John’s body, and the distance between his hand and the doorknob that result in a discharge. The more charge on John’s body the farther his hand can be from the doorknob while still getting shocked.
When implementing alternative input capabilities and descriptions, we wanted to ensure (1) leg and arm interaction is intuitive – since these are the primary interactive objects in the sim, (2) efficient access to electron transfer and discharge scenarios, and (3) descriptions support users in knowing the relative location of the arm/leg.
In collaboration with the Inclusive Design Research Centre at OCAD University, all interactive objects in the sim were instrumented to support alternative input access and to provide description. Here we focus on the navigation and description capabilities associated with the arm and leg. When the sim was first developed, the arm and leg objects were not implemented as HTML elements. The arm and leg each spanned a range of locations – 180° of a circle for the leg, and 360° of circle for the arm. We investigated implementing the arm and leg as sliders, number spinners, and in an application mode. Upon consideration, we concluded sliders would likely provide the most intuitive experience to the desired interaction pattern .
Our initial implementation of these objects as sliders had the arm implemented with a range of 0–60, and the leg with a range of 0–30, each with a step size of 1. Descriptions indicated 12 different regions of the slider (Fig. 5, left panel), with each region spanning five input values. Eleven of these regions had one description that would be read aloud, indicating proximity to the doorknob, e.g., “Position 12, Close to the doorknob” and “Position 55, Far from the doorknob.” In the region nearest the doorknob, additional descriptions were provided, indicating “Close to doorknob,” “Very close to doorknob,” and “Closest to doorknob.” Descriptions for the leg, which moves 180° of a circle, had three regions – each indicating whether foot was off (e.g., “Position 2, Foot is off the rug”), or on the rug (e.g., “Position 15, Foot is on the rug”).
The slider implementation successfully supported access to incrementing and decrementing arm and leg position values. The interaction did not support efficient access to charge transfer or discharge behavior, and the descriptions were not always meaningful during interaction. A common complaint from users was that it took too many keypresses to reach a new description region, resulting in repetitive descriptions and an overall feeling that the arm/leg was moving too slowly. Also, each object has a ‘target’ area, a location that has unique consequences for the slider interaction. For the arm, this is typically the region where the hand is nearest the doorknob; for the leg this is the region where the foot is on the rug.
Additionally, there are two challenges inherent in using a linear slider to represent a circular interaction. First, there is no way to continuously increment the input values to result in multiple same-direction (e.g., clockwise) passes around the circle. For example, incrementing the slider from input value 0 to 60 results in moving the arm clockwise around the circle. Once the maximum input value of 60 is reached, the user must stop, or decrement through decreasing input values (moving arm counterclockwise) to continue moving the arm. The second challenge is that at some point in traversing the slider input values, using the Up/Right (or Down/Left) Arrow key to increment (or decrement) the input value will actually result in decrementing (or incrementing) the position value.
To address these challenges, we decreased the input value ranges for the arm and leg, and made significant changes to the description approach. For the arm, the input value range is now −15 to 15; the leg has an input range of half that, −7 to 7. This results in requiring fewer keypresses to span the full range, and more object movement with each keypress.
To improve the descriptions of the arm position, three description types were created (Fig. 5, right panel) the region (e.g., “Close to doorknob,” “Not so close to doorknob,” “Far from doorknob”), landmark (e.g., “Hand pointing straight up” and “At doorknob”), and direction change descriptions (i.e., “Away from doorknob” or “Towards doorknob”). Upon change of position for the arm, the slider value is announced as well as one of the following: If change of direction, direction is announced; if arm enters landmark location, landmark is announced; otherwise, region is announced. The leg descriptions were also improved. For the leg, one of two regions are announced in addition to slider value: “Foot rubbing on rug” or “Foot off rug”.
The resulting sim interaction provides efficient access to the leg and arm as well as the important electron discharge scenarios, while the descriptions provide new information on relative position with each move of the leg or arm. The result is an interaction that highlights the significance of the location of the arm with respect to the doorknob, supporting the user to create and explore electron discharge events.
3.3 Providing Custom Control and Using Visual and Non-visual Cues
The sim Balloons and Static Electricity (described in Sect. 2, shown in Fig. 1) allows students to investigate static electricity, charge transfer, attraction, repulsion, and induced charge through free range movement of a Balloon. The Balloon can be rubbed on a Sweater or a Wall, resulting in different charge transfer or induced charge behavior. The Balloon can be moved by the user, and can also move independently upon release – with the Balloon location, state (more or less negatively charged), and release location all being significant for the free-movement velocity of the Balloon, and the final resting position of the Balloon.
When implementing alternative input capabilities and descriptions, we wanted to ensure students could (1) intuitively pick up, move, and release the Balloon, (2) easily understand the specific location of the Balloon, and (3) understand when a transfer of negative charge occurs.
The Balloons and Static Electricity sim presented numerous challenges for implementing accessibility features. The Balloon in the sim needs to have a four-way, drag and drop-like, behavior – with any location in the sim being a possible release location. Upon release the Balloon may move to other locations based on the current state of charge differences in the sim. This behavior is unlike any native HTML elements . Additionally, while the Balloon is in motion (either controlled by the user or moving independently) the user needs to be kept up-to-date on various state changes, including the Balloon’s current location, any occurrence of charge transfer and the resulting net charge, any occurrence of an induced charge at the wall (which occurs when the negatively charged Balloon is near the Wall), and current location and relative speed (when in motion) of Balloon release. The amount of dynamic information that happens outside the user’s keyboard focus that must be conveyed to the user is far beyond what might be conveyed by alerts in a typical web page.
The Balloons and Static Electricity sim underwent far more implementation iterations than the sims previously described. An example of an early implementation included: the Balloon implemented as a custom element using the ARIA application role with associated instructions on how to move it read out automatically, followed by the Remove Wall button, Two-balloon Experiments checkbox, and the Reset All button. The radio button group controlling charge views was not made accessible until later in the design process. In this early implementation, interaction with the Balloon involved moving focus to the Balloon and then hearing not only its label, “Yellow Balloon”, but hearing its role “Application”, its technical interactive state, “draggable”, and finally a description of how to move it, “Grab and drag Balloon up, left, down, or right with the W, A, S, or D key.” Once users moved the Balloon, they would hear alerts that would guide them towards an object (Sweater or Wall), simultaneously providing direction, progress, and actual location. They would hear the following five alerts as they progressed towards the sweater with successive presses of the A key: “Left. Towards sweater”, “Left. Closer to sweater”, “Left. On left side of Play Area”, “Left. Near sweater”, “Yellow Balloon has a few more negative charges than positive charges. Sweater has a few more positive charges than negative charges.”
These early implementation approaches resulted in multiple challenges, particularly for users with visual impairments. These challenges included difficulty recognizing the Balloon as the most important sim object, difficulty understanding how to interact with the Balloon, and difficulty recognizing the location of the Balloon during interaction.
When interactive HTML elements are accessed while using screen reader software, the user hears both the label of the interactive element (i.e., its accessible name) and the elements functional role. Ideally, the label describes what an element is or does in context, and the role reveals to the user how they will interact with the interactive object. Exploring an early prototype for the accessible Balloons and Static Electricity sim using only the Tab key (keypress indicated by [Tab]), would sound like the following, with some variation depending on screen reader software used: [Tab] “Application, Yellow Balloon, draggable”, [Tab] “Remove Wall, button”, [Tab] “Two-Balloon Experiments, checkbox, not checked” and [Tab] “Reset Balloon, button.” If the user did not move on after hearing the label and role, further descriptive content (e.g., help text for how to move the Yellow Balloon) may be read out, depending on the screen reader software in use.
Initial implementations of the Balloon resulted in users not recognizing the Balloon as an important object to interact with, and not understanding how to interact with the Balloon. The interaction required users to hear the Yellow Balloon element’s role and state first (i.e., “Application, draggable”), which does not indicate to the user how to interact with the Balloon. The more useful information, the help text indicating how to grab and move the Balloon, followed this role and state information. Most users chose to move focus to the next interactive element, rather than taking the actions described in the help text. The more inviting interaction, based on its description, was the “Two-Balloon Experiments” checkbox, resulting in most users selecting the checkbox before interacting with the Yellow Balloon. This choice is undesirable for multiple reasons as this selection made the sim screen and potential interactions more complex (with two Balloons to interact with) before users had a chance to explore and make sense of a simpler (single Balloon) scenario.
Once users successfully began interacting with the Yellow Balloon, users often ran into challenges understanding the location of the Balloon, due to the original structure of the location alerts. Initially, each keypress used to move the Balloon resulted in an indicator of the Balloon’s direction of motion. For example, when repeatedly pressing the A key to move the Balloon to the left, the initial word of the alert read out was always the word “Left.” If the user made successive soft keypresses (pressing lightly), the Balloon would not change in position enough to get a full change in location description, and would hear only the direction indicator, for example “Left. Left. Left.”. If users made harder keys presses (advancing further) and also did so quickly, the alerts would interrupt themselves, reading out, “Left. …. Left. …. Left. …” In both cases, soft keypresses or quick successive key presses, users did not get a sense of progress or a sense of where the Balloon actually was in relation to the Sweater or Wall.
To address the challenge of users having difficulty interacting with the Balloon, we implemented a two-step process for Balloon interaction. Users first “grab” the Balloon through selection of the “Grab Yellow Balloon” button, a native HTML interaction. Once the Balloon is “grabbed,” the user receives the information that it is an “Application” and the following help text “Press W, A, S, or D key to move Balloon up, left, down, or right”. While this two-step interaction has been more intuitive for non-visual users who access the description content, this interaction is less intuitive for those relying solely on the visual representations in the sim. We added visual cues (Fig. 6) that appear during initial interaction with the Balloon to provide visual guidance for this two-step interaction. With carefully designed auditory description as well as visual cues, the two-step interaction with the Balloon simultaneously supports visual and non-visual access.
To cue users with visual impairments that the Balloon is the most important object to interact with, we added two interaction hints to the PDOM description. The first is placed at the end of the Scene Summary that describes the sim on page load, “Grab balloon to play.”, and the second, “Look for grab button to play.” is placed at the end the Balloon’s description, just before the “Grab Yellow Balloon, button”.
To ensure users are kept up-to-date on the location of the Balloon during user-controlled motion and upon release, we implemented a sophisticated alert structure that still provides the user with directional information, a sense of progress and actual location, but does so more efficiently and ensures that the start of most alert text is unique to avoid repetition. After the Balloon is grabbed, a series of moving alerts that occur with successive presses of the A key now sounds like this, “Left. Closer to sweater”, “On left side of Play Area”, “On left side of Play Area”, “Near sweater”, “On sweater. Yellow Balloon picks up negative charges from sweater”, and finally, “Yellow Balloon has a few more negative charges than positive charges. Sweater has a few more positive charges than negative charges.”