Keywords

1 Introduction

With the advance of technology, control by hand gestures has been applied in our daily lives. This is evident in the cases of controlling a smart phone with gestures on the surface of its touch screens and controlling a smart TV with gestures in midair recognized by its sensors.

Two major applications of gestural user interfaces were identified in this paper as “gesture-on-surface” and “gesture-in-midair.” Gesture-on-surface refers to gesture control on touchscreens. Common applications found in literature are mobile phones, tablet computers, and public information kiosks with different sizes of display screens and angles of inclination. The tasks performed in previous research involved object selection and manipulation, text and digit entry, and function control commands. These tasks could be performed by one or more fingers with one or two hands, or by a stylus held by hand. On the other hand, gesture-in-midair denotes that hand gestures are performed without touching anything. The gestures are usually detected through cameras, gyroscopes, or motion sensors. Applications studied in literature are interactive televisions, video games, and smart appliances. Same as the research on gesture-on-surface, the tasks performed were object selection and manipulation, text and digit entry, and function control commands. These tasks could be performed by one of the three main types: freehand, handheld, and hand-worn. Freehand gestures were often the gestures made in air with an exception of using a finger of one hand as a pointer to pointing the palm of the other hand as a control panel. Handheld means performing gestures by holding a device, such as a mobile phone or the Wiimote controller. Hand-worn is to performing gestures by wearing a device, such as a ring or bracelet. Freehand gestures could be performed by one or two hands, whereas handheld and hand-worn gestures were often performed one-handed.

Most studies in the field of gesture control have only focused on young users. However, according to the statistics report of World Health Organization (WHO), the quantity and proportion of aging population has been increased rapidly around the world. In some developed countries, the percentage of old people has reached 21 % (World Health Organization 2015). Therefore, it is important to further explore the potential opportunities and challenges of gestural user interface design for elderly people. This paper is the first step of this attempt by reviewing related literature on control with hand gestures by older users. Twenty-four journal and conference papers published from 2009 to 2015 were included in this review. The remaining part of the paper proceeds as follows: Sect. 2 presents issues about age and its related characteristics. The findings from reviewed literature were then summarized in Sect. 3. Finally, a discussion was provided in Sect. 4.

2 Age and Related Characteristics

There is no standard definition of old age. The United Nations has agreed that the 60 years old is as the beginning of old age, whereas the onset of old age is delayed until 65 years old in most of the developed countries, and is advanced to 50 years old in a World Health Organization (WHO) project in Africa. As can be seen from the data in Table 1, the ages of older participants in our reviewed 24 papers varied from one to another paper, but they were within the range of 52 to 91 years old.

Table 1. Summary of reviewed papers
Full size table

Old people not only increase in chronological age but also relate to various degrees of deterioration in abilities of perception, cognition, and psychomotor (ISO/IEC Guide 71 2001). Old population also differs from other age groups in terms of prior experience and knowledge, social status, and interpersonal relationship. Seventeen out of the reviewed 24 papers collected data about participants’ background information including previous experience on the use of electronic products, physical and mental conditions, education levels, or job status.

There has been a gradual decrease in older participant’s experience of using newly-introduced technologies. Only few papers reported their older participants had no experience of using computers (Hwangbo et al. 2013; Leonardi et al. 2010), whereas older participants in the other reviewed papers had experience in using computers (Chung et al. 2010; Ferron et al. 2015; Findlater et al. 2013; Gerling et al. 2013; Kobayashi et al. 2011; Mihajlov et al. 2014). For the technologies of gesture-on-surface and gesture-in-midair, low proportions of older participants with related experience were reported in the majority of the papers (Bobeth et al. 2014; Gerling et al. 2012, 2013; Kobayashi et al. 2011; Mihajlov et al. 2014; Sáenz-de-Urturi et al. 2015; Wulf et al. 2014). However, higher percentages of older participants with the experience of gesture-on-surface were found in very recent studies (Ferron et al. 2015; Gao and Sun 2015). From results of interviews, Page (2014) found that older participants’ experience of using mobile phones was affected by their previous experience. However, there was no significant effect of older participants’ previous experience on the results in the study by Hwangbo et al. (2013).

Older participants’ sensory conditions and mobility were examined in some papers, such as vision (Chung et al. 2010; Kobayashi et al. 2011; Lepicard and Vigouroux 2012), hearing (Kobayashi et al. 2011), and the abilities of hands and upper-extremities (Gao and Sun 2015; Gerling et al. 2013; Lepicard and Vigouroux 2012; Mihajlov et al. 2014; Pham and Theng 2012; Wulf et al. 2014).

Participants’ education levels were reported in some papers (Leonardi et al. 2010; Gao and Sun 2015). Gao and Sun (2015) speculated that, compared to participants with lower education level, the participants with higher education levels might be more familiar with and have more positive attitude to new technologies. The data about the status of retirement had also been collected (Stößel and Blessing 2010). Ziefle and Bay (2005) argued that participants’ perceptions about high-tech products were influenced by whether they were retired or not.

3 Summary of Findings in Previous Research

The findings of previous research were summarized under five headings: Older Vs. Younger Age Groups, Control Tasks and Devices, Sensory Feedback, Physical Abilities, and Cognitive Abilities.

3.1 Older Vs. Younger Age Groups

The differences in gesture control between older and younger user groups have been found in a number of respects. McLaughlin et al. (2009) revealed that the attention resources of old users were affected more than the ones of young users by the design of control. Gerling et al. (2013) found that old users experienced difficulty in controlling directions with the midair gestures, and this caused the increase in their mental workloads. On the other hand, Stößel and Blessing (2010) found that the control gestures with multi-fingers were used less by old people than by the young. In addition, control gestures for the rotating and zooming of an object on touchscreens were more difficult for old users than for the young (Gao and Sun 2015; Lepicard and Vigouroux 2012). By applying 42 touchscreen control gestures with different levels of complexity, Stößel et al. (2010) found that young participants performed faster than old participants, but there was no significant difference in accuracy between the two groups. Similar result was reported in the study by Bhuiyan and Picking (2011). They grouped particiants according to their ages and found that the younger the group was, the less time they needed to complete the control tasks with midair gestures, but all the groups could perform the tasks without any error.

In summary, gesture-on-surface may be a better means of control compared to other conventional control methods, especially for older people (Findlater et al. 2013). For gesture-in-midair, old users seemed to need more instructions or hints for operation (Bobeth et al. 2014; Rice et al. 2013).

3.2 Control Tasks and Devices

The performance of older users was affected by different control tasks and devices. McLaughlin et al. (2009) reported that old users would benefit from the matching tasks, such as using a touchscreen for pointing and ballistic tasks but not for the dismatching repetitive and precision tasks. Findlater et al. (2013) found that old users performed better by gestures on touhcscreens than by conventional control devices such as mouses. Bobeth et al. (2014) showed that old people could effectively and efficiently perform gsture-on-surface control, but they were more interested in gesture-in-midair control since it could get more excercise. For gesture-in-midair, Pham and Theng (2012) compared freehand control to handheld control and revealed that handheld control was easier to be performed than freehand by old users. Furthermore, Gerling et al. (2013) demonstrated that older user preferred handheld control to freehand control since they liked the feeling of something being held during the control.

In general, elderly persons could do gesture control with ease on touchscreens for simple tasks, such as dragging or tapping an object, but not for more complex tasks, such as rotating or resizing an object (Findlater et al. 2013; Gao and Sun 2015; Kobayashi et al. 2011; Lepicard and Vigouroux 2012; Mihajlov et al. 2014; Stößel and Blessing 2010; Wulf et al. 2014). Some research results showed that control with single finger might be more suitable than with multi-finger for older users (Lepicard and Vigouroux 2012; Mihajlov et al. 2014; Stößel and Blessing 2010; Wulf et al. 2014). However, Kobayashi et al. (2011) pointed out that older users preferred multi-finger control to single-finger control if they received appropriate instruction of operation.

For gesture-in-midair, direct and simple manipulation with gestures was suitable to old people (Bobeth et al. 2012; Gerling et al. 2012). For gesture-on-surface, bigger object size (Kobayashi et al. 2011; Hwangbo et al. 2013; Gao and Sun 2015), larger screen (Stößel et al. 2010; Kobayashi et al. 2011), and vertical but not horizontal device orientation (Wulf et al. 2014) could enhance old users’ performance.

3.3 Sensory Feedback

Sensory feedback is important for old people to do the control with gestures. Chung et al. (2010) demonstrated that the performance of old users have been enhanced as virtual keyboard being located near to the output display on the screen. Kobayashi et al. (2011) concluded that visual feedback could reduce error rates. Gerling et al. 2013 compared two game consoles and found that elderly players could play more efficiently with the one showing players’ locations on the screen than the other without this feedback.

In addition to visual feedback, auditory or haptic feedback has been suggested for gesture control (Chung et al. 2010; Hwangbo et al. 2013). Hwangbo et al. (2013) suggested including both auditory and haptic feedback. However, for single type of feedback, auditory feedback was better than haptic (vibration) feedback for old people using smart phones since it might be difficult for old users to hold the phones during vibration. For gesture-in-midair, Sáenz-de-Urturi et al. (2015) recommended adding auditory feedback. Freehand type of control appears to provide less haptic feedback so it was not as preferred as handheld type of control (Gerling et al. 2013).

3.4 Physical Abilities

Physical abilities of older users affect their performances of control with gestures. Page (2014) found most of the older participants could not perform the scrolling task on touchscreens and this might be due to the physical design of user interfaces. Older users also had difficulty to do the object rotating on touchscreens since it demanded higher level of physical coordination (Mihajlov et al. 2014). Lepicard and Vigouroux (2012) reported that older participants seemed to sacrifice speed for accuracy about the gesture control on touchscreens. Moreover, this paper showed that multi-finger gestures led to more deviation from standard actions than single-finger gestures. Several studies noticed that a slow speed of performing a gesture for older users might be due to the decline of their physical abilities (Bhuiyan and Picking 2011; Chung et al. 2010; Hwangbo et al. 2013; Stößel et al. 2010). Besides, older participants were easier to feel fatigue than younger ones during their performance (Lepicard and Vigouroux 2012), especially performing gestures in midair (Gerling et al. 2012).

3.5 Cognitive Abilities

Instructions or hints about operation procedures were found necessary to assist older users in gesture control for both gesture-in-midair (Bhuiyan and Picking 2011; Marinelli and Rogers 2014; Rice et al. 2013) and gesture-on-surface (Bobeth et al. 2014; Gao and Sun 2015). Page (2014) argued that the failure of operation of older participants might because of a lack of related experience and knowledge, as well as the deterioration of working memory. Therefore, the operation procedures should be simple and intuitive (Bobeth et al. 2014; Ferron et al. 2015), such as single-finger gestures (Mihajlov et al. 2014; Stößel and Blessing 2010) or gestures without the need to learn (Bobeth et al. 2012; Gao and Sun 2015). For example, the dragging gesture on touchscreens (Mihajlov et al. 2014). The speed of older user’s performance might be influenced by the decline of working memory (Chung et al. 2010; Hwangbo et al. 2013). Also, the performance seemed to be impacted more by the effect of cognitive abilities than physical abilities (Hwangbo et al. 2013).

4 Discussion

This paper has reviewed literature related to gesture control by older users. Suggestions of gestural interface design have been summarized as follows:

Considering the decline in mental and physical abilities and various backgrounds of older people, gestural interface should be simple and intuitive. While the term of simple means less physical and mental effort to perform gestures, and fewer steps to be memorized or learned, the term of intuitive refers to meet users’ expectations according to their previous experience and knowledge. More specifically, gestural interface should be designed with suitable number, size, spacing and layout of objects on display screens for older users to manipulate them directly under their own pace. The mappings between control commands and corresponding gestures should be designed to reflect older users’ previous experience and abilities in perception, cognition, and psychomotor. Physical demand of gestures, especially gesture-in-midair, should be examined to avoid older users suffering from fatigue. Finally, control instruction and feedback should be considered in the design of gestural interfaces.

Chronological age is not the only indicator to define whether a person is old. Further research should collect more data about participant’s backgrounds, such as previous experience, mental and physical abilities, and social status, so that the cause-and-effect relationships between age-related attributes and performances of gesture control can be identified and applied to the design of gestural interfaces.

This review of literature concludes that research on gestural interfaces for elderly people is worthwhile for further exploration. Research on the design of gestural interfaces, control tasks, devices, and gesture types, as well as the evaluation of user experience and usability is recommended.