Keywords

1 Introduction

Muscle fatigue is the transitory decrease in a muscle’s capacity to contract and perform actions, which leads to the development of musculoskeletal imbalance if it persists for longer stretches of time and overruns the body’s recovery system [1, 2]. Previous work has reported on subjective perceptions of fatigue [3, 4], which are deemed unreliable for assessing physical demands [5], and objective measures of muscle fatigue [6, 7] on tablet sized vertical touchscreens. Past research has also explored how display angle affects a range of different aspects of users’ interaction with large interactive displays (e.g. [8, 9]). However, relatively little attention has been paid to the impact that display angle has on users’ physiology over time and consequential musculoskeletal damage. The few studies that have investigated this issue have required subjects to perform a constrained task repeatedly for a relatively short time, rather than exploring the typical varied and longitudinal usage that designers envisage, such as teachers’ interaction with interactive boards (e.g., [10] who looked at vertical boards only). Other studies, such as [11, 12], have reported informal and conflicting insights for longitudinal usage of large interactive displays.

We present a study that measured muscle fatigue objectively and subjectively for both horizontal and vertical large interactive displays using an unconstrained task, which required users to perform gestures commonly used in multi-touch interaction. Our results suggest that when designing multi-touch applications that will be used for extended periods, designs that consider fatigue should, in general, favor horizontal over vertical displays, and in both cases should aim at locating interactions at closely accessible locations on the display whenever possible. Our results also suggest that designers should pay close attention to objective measures of fatigue when evaluating the appropriateness of their designs to users’ physiology, rather than relying on subjective measures alone.

2 Study Design

We performed the study on a Microsoft PixelSense multi-touch tabletop with a 30” rear projected display (21” high, 27” deep, and 42.5” wide). The task was to complete a series of 25-piece puzzles, using the tabletop’s Puzzle application, which was adapted to include scaling of pieces. Pieces could be translated, scaled and rotated, or an integrated combination of two or more of these actions, using one or more fingers, uni- or bi-manually. Initially, the pieces of each puzzle were displayed at random orientations, sizes and locations. Two pieces could only connect to form one larger piece if they were a similar size (within 20 %) and orientation (within 30°). Once completed the puzzle was replaced with another. This exercise was repeated for one-hour. The puzzles were displayed in a random order and subjects completed five puzzles on average. This task was chosen because it required participants to complete common types of interactions used when performing different tasks using large interactive displays, along with intermittent periods of no activity.

3 Procedure

Surface Electromyography (SEMG) was used to monitor three muscles on each arm before, during and after the task. These muscles are: middle deltoid, bicep brachii, and extensor digitorum muscles (see Fig. 1a). Lozano et al. [7] suggested these muscles are appropriate sites for measuring gestural interaction and fatigue on a multi-touch tablet when using SEMG. The deltoid placement was altered, from anterior to middle, to consider both shoulder abduction and flexion. To locate the placement of the SEMG sensor for each muscle, a specific location on the arm was palpated as the subject performed a movement activating the target muscle. The skin was then shaved, wiped with alcohol and the sensors placed parallel to their prospective muscle at 2 cm apart. We followed Criswell’s guidelines [13] to ensure correct placement.

Fig. 1.
figure 1

The experimental set up for the horizontal (a) and vertical (b) configurations

Full size image

Two measures were then collected: maximal voluntary isometric contractions (MVIC) and ratings using the Borg scale [14] both before and after the task (as described below). The subjects were then asked to sit and familiarize themselves with the task as the possible interaction techniques were described verbally. They were then asked to wait for the start and the end of the session to be announced. During the session, ten electromyograms (EMG) were recorded for muscle activity analysis. Participants were all males to limit interferences due to anatomical differences [15].

3.1 Experiment 1: Horizontal Configuration

Eighteen males, who were familiar with multi-touch technology and were free of musculoskeletal disorders, took part. The mean age was 26.4 (± 4.6). Three were left-handed. The tabletop was raised to a height of 26” using wooden panels to allow comfort and participants sat on a chair 18” in height (Fig. 1a).

3.2 Experiment 2: Vertical Configuration

A second participant group of eighteen males took part; all were right-handed, familiar with multi-touch technology and free of musculoskeletal disorder, with an average age of 28.8 (± 4.8). The tabletop was placed on its side on top of a desk and participants were sat on a chair 18” in height (Fig. 1b).

4 Objective Measures

Muscle fatigue has objectively been detected non-invasively using sonomyography, near-infrared spectroscopy, mechanomyogram, and SEMG. Recent research has also proposed a novel approach, “Consumed Endurance” [16], which does not necessitate the employment of specialized equipment. A survey carried out by Al-Mulla et al. [17] found SEMG to be the most suited measurement for the detection and quantification of fatigue. Previous applications of SEMG for the assessment of fatigue include interacting with software on a traditional setup [18] and a multi-touch table [6, 7]. Accordingly, SEMG was used to detect and measure physiological changes to skeletal muscles due to contraction. Processed EMG data can provide information about localized fatigue and force.

A MVIC, where the muscle’s tension changes while its length remains the same [19], is used to quantify force and localized muscle fatigue. To gain a MVIC the subject is instructed to achieve the greatest possible force of contraction, constantly, for a short period. To do this our subjects were asked to remain seated and maintain a posture for 10 s while holding a 2.5 kg weight. The postures were; (i) Middle deltoid: arm elevated at 90° in the frontal plane; (ii) Bicep brachii: arm held close to the body while elevating the forearm at 90° in the sagittal plane; (iii) Extensor digitorum: the forearm resting on a desk with the wrist resting on the edge of the table. A MVIC is a baseline and so future readings may be greater than 100 %.

Localized Fatigue.

For fatigue indexing, we adopted the median power frequency (MPF) of the MVIC in the time domain as a reference point [20]. Essentially, a decreasing MPF signal indicates that muscle fatigue is increasing. Previous research corroborates the reliability and consistency of this method of analysis (e.g. [21]) unlike using the amplitude in the time domain, where the literature reports significant contradictions (e.g. [7, 21]).

Force.

‘Force’ quantifies a muscle’s electrical activity during contraction and is described as a percentage of a MVIC. To extract force information we integrated a rectified EMG, a long-established technique due to their linear relationship [22].

The SEMG used was ZeroWire, a wireless system with six surface channels, with Biosense’s bio-logic disposable press-stud electrodes (Ag-AgCI). This system operates using light autonomous signal processing and power transmission units, each weighing 10gm. Each channel provides a bandwidth of 10-1000 Hz for a signal sampled at 2000 sample/sec. The transmitters wirelessly transfer the signals captured with the electrodes to the main unit, which is directly connected to a computer running the ZeroWire software suite. This minimizes the restriction of a user’s movements. Matlab and Microsoft Excel were used to process the EMGs and analyze the results.

4.1 Localized Fatigue Analysis

The 10-second MVIC collected before and after the task was divided into 10 segments with a 50 % overlap. This led to 3000 data points for each segment from the original 2000. The last segment was excluded from the analysis because of the overlap. Each segment was then rectified and passed through a low pass fourth-order Butterworth filter with a cut-off frequency of 500 Hz. A fast Fourier transform was then performed to calculate the power spectrum of each segment from which the MPF is obtained. Matlab was used to process the EMG using functions provided by the Signal Processing Toolbox (DSP) and the Biomechanics et al. Toolbox (BEAT) [23].

For normalization (to eliminate variations between the subjects such as age and muscle mass) the data recorded before the task was averaged and used as a reference value for the data collected after the task. These values were averaged to produce singular values representing the MPF of each muscle. A paired t-test was then carried out between the two data sets.

4.2 Localized Fatigue Results

Horizontal.

While some muscles did show a decrease of MPF indicating some level of fatigue, none of them showed a significant evidence for increased fatigue. The dominant and non-dominant extensor digitorum showed a decrease to 80 % and 63 % respectively, while the dominant and non-dominant bicep brachii showed a decrease to 88 % and 95 % respectively. Moreover, while the dominant middle deltoid also showed evidence of significant increase of MPF indicating decreased fatigue (119 % and t(17) = 1.74, p = 0.01), no change was noted for the non-dominant middle deltoid.

Vertical.

The middle deltoid showed a significant decrease of MPF indicating increased fatigue (63 % and t(17) = 1.74, p = 0.003 for the dominant, and 71 %, t(17) = 1.74, p = 0.02 for the non-dominant side). As for the extensor digitorum only the non-dominant side showed a significant decrease of MPF indicating increased fatigue (75 % and t(17) = 1.74, p = 0.01) with no noted decrease for the dominant hand. The biceps brachii showed non-significant decrease of MPF for the non-dominant hand to 79 % and no-decrease for the dominant hand.

4.3 Force Analysis

Ten 1-minute EMGs were collected during the experimental task at 5-minute intervals. Each EMG was passed through a low-pass fourth-order Butterworth filter with a cut-off frequency of 500 Hz. The averaged root mean square was then calculated using BEAT, which was then averaged. Normalization was carried out using the averaged MVICs collected before the start of the task. The normalized values were then averaged to represent the muscle activity as a percentage of the MVIC (see Fig. 2). A one-way repeated measure analysis of variance (ANOVA) was then carried out.

Fig. 2.
figure 2

Force results for 1-minute readings collected every 5 min for both configurations

Full size image

4.4 Force Results

Horizontal.

A one-way repeated ANOVA for the 10 measures showed that only the dominant extensor digitorum showed significant inconsistency of muscle activity throughout the task (a change of 13 % and F(17, 9) = 3.09, p = 0.002). No significant evidence was found against the other muscles proving their activity to be consistent. Nevertheless, the non-dominant extensor digitorum showed an increase and decrease of up to 8 % and the bicep brachii showed a change of up to 10 % and 14 % for the dominant and non-dominant sides respectively. The middle deltoids’ activity proved relatively stable with activations ranging from 8-11 % for both sides.

Vertical.

A one-way repeated ANOVA for the 10 measures showed significant inconsistency of muscle activity throughout the task for the dominant extensor digitorum (a change of 13 % and F(17, 9) = 2.85, p = 0.004). No significant evidence was found against the other muscles proving their activity to be consistent. Nevertheless, both biceps brachii showed an increase and decrease in activation of up to 13 %, while both middle deltoid muscles showed relatively consistent activation of up to 18 %.

Table 1 marks the muscles that showed significant evidence of increase in level of fatigue after the trial session for both configurations. It also marks significant inconsistencies in activation of force during task interactions.

Table 1. Objective measures summary of significant increase in fatigue and force inconsistency
Full size table

5 Subjective Measures

When examining fatigue, subjective ratings are commonly used to assess perceived physical exertion and supplement objective measures. We used the Borg CR100 scale [14], which is a fine graded ratio scaling method that estimates the level of exertion and determines the ratio relationship between perceptual responses. The scale contains subjective dynamic ranges with values ranging from 0 to 120 annotated with verbal anchors ranging from “nothing at all” to “absolute maximum”. This scale was chosen as the most appropriate after critical assessment and comparison of four other scales. The subjects were asked to rate their perceived muscle exertion after the MVIC had been recorded before and after the experiment.

The average of the scores collected was analyzed for statistical significance using a paired t-test. For the horizontal configuration, the averaged value of the ratings was significantly greater after the task (t(17) = 1.74, p = 0.0003). Similarly for the vertical configuration, the averaged value was significantly greater after the task (t(17) = 1.74, p = 0.0005). This indicates an overall increased perceived level of fatigue for both configurations (see Table 2).

Table 2. Subjective mean and standard deviation results for the horizontal and vertical configurations
Full size table

6 Discussion and Conclusion

The main aim of the study was to compare the muscle fatigue experienced when using large multi-touch interactive displays in horizontal and vertical configurations, over a one-hour period.

Our findings from the objective measures provide statistically significant evidence for the presence of potentially damaging fatigue for the middle deltoids for the vertical case only and not the horizontal. This can be due to the elevation required of the arms when interacting with the vertical display (i.e. gorilla-arm effect [24]) and the employment of larger force activation percentages. The presence of fatigue was also noted for the non-dominant extensor digitorum despite consistent activation of force. This can be due to the regular use of the non-dominant muscle (either uni- or bi-manually) in the vertical configuration, as can be derived from the significantly inconsistent activation of the dominant extensor digitorum. The inconsistent activation of dominant extensor digitorum suggests that the users were alternating between dominant and non-dominant sides. Unlike the objective measures, the results of the subjective measures showed that the subjects perceived the presence of fatigue in both the vertical and horizontal configurations with statistical significance. This emphasizes the unreliability of a person’s perception of fatigue, where objective results found significant levels of muscle fatigue that can be damaging only in the vertical case.

These results have clear implications on interaction design for large interactive displays. Vertical interactive displays, while suitable for intermittent use over short periods of time, are not as suitable for frequent longer use – our study showed presence of fatigue for a one-hour task, but it might occur in shorter durations. Prolonged activities leading to muscle fatigue, which has previously been reported anecdotally, have the potential to lead to musculoskeletal disorders [1, 2]. For tasks that require frequent interactions in the range of one or more hours, designers should choose a horizontal display where possible to minimize damage to the musculoskeletal system, modify the design of interaction techniques to reduce the need for continuous interaction, or to change the location of distant interactions when possible to more accessible locations on the display. Moreover, subjective measures showed a significant level of fatigue for both horizontal and vertical cases, indicating that for either case, designers should locate points of interaction in spaces easily accessible to the user whenever possible. As a simple example, for large displays, the use of contextual commands that are located based on users’ interaction location is recommended over the traditional desktop model of showing commands on upper or side toolbars and menus. However, identifying the display space that helps reduce fatigue, whether on horizontal or vertical displays, is the subject of further investigation.

This study has several limitations that point towards future work. We studied fatigue for only male participants to overcome physiological differences. Additional studies with female participants would give insight on sex differences and its effect on measured fatigue when interacting with large interactive displays. Furthermore, our study only considered a small subset of the muscles’ activated when interacting with the display and only two angles of interaction. In the future, we plan to assess the fatigability of other muscles (e.g. capitis muscles) at additional angles of interaction.