1 Introduction

There are about 36 million blind people and 216 million people with moderate to severe visual impairment in the world [4], a population also referred to as individuals with low vision. According to the World Health Organization, low vision is currently defined as individuals with visual acuity below 6/18 down to and including 3/60 in the better eye with best correction [24]. This number of individuals affected is expected to increase with population ageing, as the majority of people with vision impairment are over the age of 50 years. A large range of heterogeneous pathologies can be aggregated in this definition and each pathology usually exhibits different types of symptoms and may therefore require different functionalities from assistive devices.

Numerous technological assistive devices have already been developed in research projects or are available commercially, such as smart white canes, reading devices, text-to-speech, dedicated smartphone applications and mobility aids. Most of these tools focus on assisting visually impaired people (VIs) in their daily life functions. However, only few solutions focus on facilitating social interactions, which is one of the main handicaps and complaints of VIs as shown in this study. Key elements for social interaction such as face identification, visual attention awareness and facial expressions of emotion processing represent major challenges for VIs. To our knowledge, there is currently no clinically validated solution aiming specifically at these social functions. Therefore, our research group is currently developing and clinically assessing the use of smart glasses and augmented reality to assist VIs with social interactions.

Very few studies have actually investigated the concrete needs of VIs from smart glasses assistive technologies. These studies have been performed with limited quantity of participants and using open interviews or custom questionnaires not made publicly available [28, 29]. To our knowledge, no study has yet investigated difficulties and needs of VIs while considering the different types of deficit within people with low-vision. This gap in the literature is significant, as we believe defining and understanding concretely the range of needs and expectations of people with low-vision according to their pathology and symptoms is critical for researchers and developers in order to design tailored user-centered solutions.

In this study, we investigate the link between the quality of life of 50 visually impaired individuals and their type of visual deficit, their difficulties in functional and social interaction as well as their expectations for smart glasses as assistive multipurpose tools. Three questionnaires have been used for this investigation. The NEI-VFQ-25 is a standard questionnaire that has been largely used to assesses the general quality of life of VIs [23]. In order to better quantify specific difficulties and expectations, we also developed two new questionnaires in collaboration with psychologists and the Swiss Association of the Blind and Visually Impaired (SBV). The DRFE questionnaire assesses specific strategies and difficulties in recognizing faces, emotions and visual attention and the ESGAD assesses the currently used assistive technologies and the expectations regarding smart glasses as an assistive device. We notably hypothesize that VIs have different needs and expectations from assistive technologies depending on their specific pathology and symptoms. With this study, we aim at better characterizing these specific needs in order to guide future works on assistive tools for VIs.

2 State-of-the-art

The majority of assistive devices developed for visually impaired people have been focusing on supporting functional tasks such as reading, navigation and obstacle avoidance as highlighted by the survey of Tapu et al. [35]. Many prototypes have been developed in the context of academia and research, most of them relying on technical approaches, trying to leverage the potential of new technologies without relying on a proper user-centered design methodology [13, 14, 18, 38]. Commercial products that have attracted the VI community, such as OrCamFootnote 1 or eSightFootnote 2, also mostly concentrated on challenges in this area.

In their comprehensive survey, Bhowmick and Hazarika implied that researchers should pay more attention to social aspects of assistive devices [3]. Meanwhile, although novel types of assistive devices were introduced with technological advancement in the last decades, only a few of them have been accepted by users. Krishna et al. highlighted the lack of user-centered design [20], while Gori et al. reported that obtrusiveness of device along with extensive time and efforts required to learn technologies involved could be the main reasons hindering user acceptance [15]. Moreover, many of the proposed systems did not undergo clinical tests to validate their effectiveness [28]. Nevertheless, technologies are evolving rapidly and latest computing devices may support the visually impaired satisfactorily [16].

Smart glasses have received tremendous attention recently and emerged as a new platform of choice, notably due to hands-free interaction in contrast to smartphones, augmented reality (AR) capabilities and constant availability [20]. Besides, studies performed in other domains showed that smart glasses have great potential to enhance social interaction [21, 37]. Typically, assistive systems apply two different concepts to transfer information to users: sensory substitution and visual augmentation. Sensory substitution (vision substitution) is a coping strategy for visual impairment that converts visual input to auditory and/or tactile stimuli [6, 12, 22, 33]. Visual augmentation (vision enhancement) refers to enhancing vision of visually impaired people, by adding or subtracting information from their field of vision [2, 16, 18, 34, 39]. Sensory substitution technique can be highly effective to support completely blind users, but visual augmentation may be more advantageous for low vision [10]. Several researches have adressed assistive technologies for VIs that focus on functional issues in daily life. For example, Zhao et al. demonstrated that AR can effectively improve mobility (e.g., walking through stairs) for people with low vision by displaying or projecting digital figures on physical objects [40]. In their work, Hwang and Peli developed a prototype that reinforces object detection by intensifying edges in images captured through Google Glass [18]. Huang et al. used a Microsoft HoloLens device to help VIs understand visual signs like door labels [17]. Caraiman et al. proposed a system that identifies obstacles and delivers auditory or tactile feedback in order to help VIs be more aware of their surroundings when navigating [6]. Similarly, Suresh et al. developed a sensory substitution device for VIs that provided auditory or tactile signals on wearables [33]. Their project, the Smart Glass, could detect objects using deep learning algorithms and estimate distances to them through ultrasonic sensors.

Several studies showed that social interaction can be particularly demanding for VIs because communication with others includes exchanges of nonverbal cues such as gaze, gesture, or facial expression that cannot be recognized by VIs [5, 36]. Phillips and Proulx insisted that assistive devices should facilitate acquisition of nonverbal information; they defined a set of design criteria in their work: functionality, usability, cognitive demand and aesthetics [25]. According to Qiu et al., helping the blind interpret and respond to visual cues from others may lead to more successful communication [27]. Since aging is correlated with visual impairment, older adults may be especially susceptible to poor social interaction [9]. In the study of Desrosiers et al., the elderly with visual impairment showed significantly reduced social activities compared to others with normal vision [11]. Furthermore, Cimarolli et al. conducted a longitudinal study with 364 visually impaired elders to discover temporal changes in their difficulties finding that challenges for social events (e.g. eating out or recognizing people) persist consistently or become worse in a long term [8]. As previously mentioned, few studies in the field of assistive technology have attempted to foster social interaction for VIs. Chaudhry and Chandra proposed a facial detection and recognition system to assist visually impaired people [7]. Using a camera from an Android device, their system could distinguish faces relying on an internal database or online server. The success rates for facial detection and recognition were 93% and 70%, respectively. Similarly, Zhao et al. implemented a smartphone-based prototype, the Accessibility Bot, that could detect faces of friends on Facebook messenger [41]. It also provided supplementary information about people around the user including their locations and facial expressions. Although the system was not always easy to use in real-world settings, it had a positive impact on social interactions of the participants. Sarfraz et al. worked on a wearable multimodal system to help VIs understand non-verbal cues via auditory and tactile feedback [31]. They built a prototype that consisted of a body-worn camera, haptic-belt and bone-conduction headset to inform users of the identification and position as well as visual attention of other people nearby. The evaluation performed with 12 VI persons demonstrated the usefulness of the proposed solution; although several subjects reported the complexity of the wearable system for a daily usage.

More recently, some studies have taken advantages of smart glasses to improve social interactions of VIs. Qiu et al. tried to reinforce the sense of involvement of blind people when interacting with others by simulating natural gaze [26]. They designed a prototype named Social Glasses that could display artificial gaze of VIs in response to the normally sighted and it promoted more engaging conversations. In our previous work, a prototype, based on the Epson Moverio BT-200 smart glasses, was developed to identify surrounding people and their emotions, the feedback was provided through sound and visual augmentation [28]. The preliminary evaluation with VIs demonstrated the feasibility of the concept and the strong interest of VIs for social assistive smart glasses. The study also highlighted remaining technical and hardware problems that should be resolved before using such system in the wild and the strong heterogeneity of expectations and needs amongst visually impaired users. Buimer et al. developed a custom assistive device for VIs that supports emotional recognition by utilizing haptic technologies [5]. In this project, six emotions captured through a camera on glasses were mapped into the same number of vibrators on a waist belt so that the user can receive tactile signals on different spots according to emotional expressions of interlocutors. Even though the authors observed limitations in differentiating emotions due to luminosity conditions or changes in angles of faces, more than half of 8 participants with visual impairment in their experiment could use the system without difficulty during conversations.

Only few studies investigated the concrete needs and expectations of VIs regarding assistive devices. The impact of the aesthetics of assistive devices has been investigated by Santos et al. [30]. They interviewed eight young individuals who knew at least one visually impaired person in order to evaluate their perception of disability. Their study concludes that devices without negative symbolism and with modern aesthetics such as smart glasses are better accepted and are less stigmatizing than more traditional assistive devices such as white canes. The work of Shinohara and Wobbrock indicated that VIs tend to avoid being identified as the disabled and stigmatized [32]. In his study, Sandnes attempted to characterize needs of VIs through directed interviews and showed that facial recognition and text reading are reported as the biggest challenges [29]. In their study, Azenkot et al. noticed that people with low vision often struggle with navigating or finding products when shopping and suggested that visual augmentation with smart glasses can benefit the target users by exploiting their residual vision [2]. The results from our prior study revealed that assistive technology for VIs has to be tailored to specific needs of the end-user [28]. For example, some may prefer visual feedback from the system because of intrusiveness, whereas others feel more secured when auditory or tactile support is given due to the severity of visual loss. Contextual adaptation of communication method may also play an important role, and as VIs often have to rely on hearing for various activities, it may not be ideal to provide auditory feedback especially in noisy outdoor settings [19, 35].

To the best of our knowledge, there have been limited studies that systematically analyzed requirements of assistive devices, and more specifically smart glasses, for visually impaired individuals with an adequate number of well-defined clinical populations.

3 Methodology

The study consisted in an interview with visually impaired individuals, in which the participants were asked to answer three different questionnaires in order to assess: (1) quality of life, (2) daily difficulties, (3) face processing strategies during social interactions and (4) expectations for smart glasses as an assistive device. The participants of the study have all been recruited on a voluntary basis through the Swiss Association for the Blind and Visually Impaired (SBV). The population consisted of 50 persons, French native speakers, aged from 28 to 87 years (\(M = 61.68, \hbox{SD} = 14.14\), ); comprising 37 women and 13 men. The population was classified into 5 distinct groups according to their visual pathology (Table 1): Age-related macular degeneration (AMD), Retinitis pigmentosa (RP), Glaucoma (Gl), Optical nerve lesion (ONL) and Others. The Others group is comprised of less common pathologies (Ocular toxoplasmosis, Hydrocephalus, Thrombosis, etc.). Typically, the potential visual symptoms vary according to the severity of each pathology; most common visual symptoms are summarized below:

  • AMD: darkening/loss of central vision, distortion, overall blurriness and sensitivity to glare;

  • RP: darkening/loss of peripheral vision, impaired night vision, loss of contrast sensitivity;

  • Gl: darkening/loss of peripheral vision, blurriness, sensitivity to glare;

  • ONL: loss of visual field (central and or peripheral), loss of color vision, flashing light;

Table 1 Distribution of pathologies amongst the studied population
Full size table

As visually impaired individuals present major differences in terms of difficulties and needs, low vision therapists insist on the importance of tailored rehabilitation and support. This aspect must be considered carefully by researchers aiming at developing smart glasses technology for this population. To our knowledge, there does currently not exist any specific scale-based questionnaire addressing those questions. Therefore, we collaborated with a study group consisting of one certified clinical neuropsychologist and two certified low vision therapists from the SBV to identify the most relevant questions that should be addressed to VIs in order to have a better understanding on face processing strategies used during social interactions (DRFE) and on their expectations towards smart glasses devices (ESGAD). This resulted in the design of a set of three questionnaires detailed below. The NEI questionnaire, already largely used by clinicians internationally [23], provides a standardized global scale as well as sub-scales for specific topics, while the two new exploratory questionnaires (DRFE and ESGAD) provide qualitative information for each addressed item (for which the use of a scale is possible only for individual questions). Those questionnaires can be reused freely for research purposes and are available onlineFootnote 3 in French and English:

  1. 1.

    NEI-VFQ 25—A standard questionnaire commonly used by clinicians to assess quality of life of VIs through 25 questions to address 12 sub-scales: general health; general vision; ocular pain; near activities; distance activities; vision specific: social functioning, mental health, role difficulties, dependency; driving; color vision and peripheral vision [23]. The questionnaire has been slightly adapted in collaboration with the SBV; the Driving subscale has notably been replaced by the Sensitivity to luminosity subscale. Individual scores are recoded and transformed on a scale from 0 to 100, where 100 represents the best possible performance and 0 the worst. An average score is calculated for each of the 12 subscales, and an average of the subscales scores is calculated to obtain the overall composite score.

  2. 2.

    Difficulties in Recognizing Faces and Emotions (DRFE) - An exploratory questionnaire to assess the functional origins of social difficulties and their impact on daily life. It consists of 14 questions, based on the 5-points Likert scale. The questions address the following topics: difficulty to recognize faces for different familiarity levels; distance at which recognition difficulty starts; negative impact on the person’s daily life; impact of the luminosity conditions; strategies to cope with deficit in face and emotion recognition and general difficulties during social interactions.

  3. 3.

    Expectations on Smart Glasses as Assistive Device (ESGAD) - An exploratory questionnaire to assess the assistive technologies currently used by the Vis and their expectations regarding smart glasses assistive devices. The questionnaire consists of 8 questions, based on the 5-points Likert scale, that address the following topics: functional and social daily difficulties, commonly used assistive devices, knowledge of smart glasses technologies, expectations from a smart glasses assistive device, desired information returned by the device on recognized persons and finally importance of comfort and aesthetic of the device.

The interviews were performed in SBV offices in Fribourg and Lausanne (Switzerland) in 2018. They followed a strict procedure, described hereafter. Upon arrival, the participants were greeted and led to a specific room for the interview. The goals of the study were briefly introduced to the participants and they were advised of the confidentiality of the study and the possibility to opt out at any time. The participants were asked to fill a consent form. Then, general information about the participants was collected: age, visual pathology, study level, profession, actual social situation. The interview continued with the questionnaires, each question and propositions of answers were read out loud by the interviewer to the participants. The questionnaires were presented in the following order: NEI-VFQ, DRFE and ESGAD. Note that the ESGAD questionnaire contains a short text introducing the concept of smart glasses assistive device for VIs. At the end of the interview, the participant had the chance to ask questions, add specific remarks and discuss openly with the interviewer. The procedure lasted approximately 45 minutes on average.

4 Results

The results for the three questionnaires are presented in their respective section below. For each questionnaire, the most relevant items are summarized for the general population and for each distinct group. This approach provides the possibility to highlight the general tendency of the studied population and the main differences amongst the considered groups. The statistics have been computed with the IBM SPSS Statistics software (Version 25). The results of the relevant items are provided with their Mean and Standard Deviation (M, SD) or their Median and Interquartile Range (Mdn, IQR) and illustrated with box plot charts or with bar charts. Nonparametric tests were used to analyze the results per visual pathology group.

4.1 NEI-VFQ-25 questionnaire

The main results of this questionnaire are resumed in Fig. 1. The reported values from this questionnaire indicate the following information; a score of 0 indicates severe or frequent difficulties for the considered item, while a score of 100 indicates infrequent or absence of difficulties. The studied population presents a low composite score (\({M = 45, \hbox{SD} = 14}\)). Their general health (M = 51, SD = 24) is low compared to the score of a standard population (M = 69, SD = 24) reported in a previous study [23]. As expected, our sample reported difficulties with general vision (M = 37, SD = 17). Their vision was decomposed into five different characteristics: moderate difficulties with near vision (M = 47, SD = 17), frequent problems with distant vision (M = 30, SD = 18), some difficulties with peripheral vision with a strong variance amongst subjects (M = 40, SD = 38), some difficulties with color vision (M = 63, SD = 26) and some difficulties regarding sensitivity to luminosity (M = 40, SD = 30). Social functioning is generally reported as a moderate difficulty (M = 62, SD = 20). What characterizes the results is the strong variability of the scores for most subscales. Additional information can be extracted by analyzing specific items of the questionnaire. The overall population seems more subject to glare in outdoor conditions (Q5: M = 34, SD = 32) compared to indoor conditions (Q6: M = 46, SD = 34). They have severe difficulties to detect obstacles on the ground in low (Q13: M = 27, SD = 26) and high luminosity conditions (Q14: M = 29, SD = 20).

Fig. 1
figure 1

Results of the whole population on the NEI questionnaire. The Composite and the 12 subscales scores are presented in the graphic as individual boxplots. The NEI Score represents the evaluation of the item by the patient (where 100 represents the best possible score and 0 the worst)

Full size image

The analysis of the results according to the distinct groups considered in the study provides additional information. The global composite score is homogeneously low for all the groups; only the Others group exhibits larger variations. The AMD and RP groups reported a better general health compared to the other groups. The results for the vision-related subscales are illustrated in Fig. 2. The following facts can be observed: the general vision score is homogeneously low for all groups, with a larger variation for the Others group. Near vision has been reported as less problematic than distance vision by all considered groups. The ONL population has large variations for distant and near vision. Peripheral vision exhibits a large variation between and within groups; as expected RP, Gl and ONL groups report severe difficulties with peripheral vision, while AMD population reports only moderate difficulties. Color vision is not reported as problematic by any of the studied groups. The sensitivity to luminosity is the subscale presenting the most variations between groups. The AMD, RP and Gl group have severe difficulties, while ONL and Others are less impacted.

Fig. 2
figure 2

Results for vision-related subscales of the NEI questionnaire presented per visual pathology

Full size image

The analysis of the results per visual pathology group provides interesting information regarding specific questions. The RP group is significantly more impaired by light outside (Q6: M = 5, SD = 26) than inside (Q5: M = 29, SD = 19), T = 15, z = − 2.07, p = .026 ; the AMD group is significantly more impaired by glare in indoor conditions (Q5: M = 25, SD = 19) compared to the rest of the studied population, t(24) = 3.12, p = .005; while on the contrary, the Gl group is significantly more impaired by glare in outdoor conditions (Q6: M = 8, SD = 13) compared to the rest of the studied population, t(12) = 3.89, p = .002. The AMD group reports severe difficulties to read a paragraph in a newspaper (Q7: M = 0, SD = 0) and to read street signs (Q11: M = 5, SD = 10); they report more difficulties on those two items than the other groups, t(15) = 4.86, p < .001. In contrast, individuals with AMD report less difficulties in seeing elements with their peripheral vision than other groups (Q15: M = 61, SD = 36), t(17) = − 2.23, p = .039.

4.2 DRFE questionnaire

The results of this questionnaire highlight the need for social assistive tools, notably with Face, Emotion and Visual Attention as illustrated in Fig. 3. Difficulties to recognize faces are reported as being a severe discomfort, while difficulties with emotions are reported as being a moderate discomfort. Participants report frequent difficulties with visual attention awareness in 1-to-1 and group conditions.

Fig. 3
figure 3

Histograms illustrating the percentage of population experiencing difficulties in their daily life for selected items. Top: discomfort due to difficulty in recognizing faces and emotions. Bottom: frequency of difficulties with visual attention in 1-to-1 and group conversations

Full size image

Most participants (68%) have difficulties recognizing faces when they are close (< 1 m) and almost all of them (98%) report difficulties at distance (> 1 m). They report difficulties with face recognition as a severe discomfort for their daily life (Mdn = 4, IQR = 2). This difficulty is present regardless of the familiarity of the person to recognize. Most participants (68%) report looking directly at the face in order to identify a person, while few (32%) report looking eccentrically. Most participants (92%) report not focusing on a specific aspect of a face during the identification process. Low and high luminosity conditions have a negative impact on the identification process for most participants (88% and 82%, respectively). Participants largely rely on voice (98%) to identify a person, followed by gait (24%) and clothes (16%). Most participants (80%) report difficulties to recognize facial expressions of emotion and this holds true for all types of emotions. However, this difficulty is reported as being only a moderate discomfort (Mdn = 3, IQR = 2). Participants also report very frequent difficulties in visual attention: knowing if a single interlocutor is looking at them (Mdn = 2, IQR = 2) and knowing if an interlocutor is addressing them during a group discussion (Mdn = 2, IQR = 1).

Fig. 4
figure 4

Selected results of the DRFE questionnaire presented per visual pathology. Left: Reported discomfort with difficulties with Face Recognition and Emotion Recognition. Right: Reported frequency of difficulties with visual attention in direct and group communications (“Frequency” scale was reversed for readability: 1=never, 5=always)

Full size image

The analysis of results per group for selected items is presented in Fig. 4. All groups report difficulties with face recognition as being a severe discomfort. The ONL group has more variations for that item; only 50% of ONL report difficulty recognizing familiar faces and 80% report difficulties with unfamiliar faces. Other groups do not report differences between familiar and unfamiliar faces. Most groups report, on average, moderate difficulties with emotion recognition. This difficulty varies largely within groups, from mild to severe difficulties. All groups report experiencing difficulties with visual attention detection frequently. AMD and ONL groups seem to exhibit larger variations for those items (from never to always) compared to the other groups. The analysis of specific questions provides additional insights. The majority of AMD (82%) and RP (79%) individual report having difficulties recognizing faces even at close distance (<1 meter), while fewer individuals from ONL (60%), Others (57%) or Gl (51%) groups report such difficulties. All subjects, except a single individual with AMD, reported difficulties to recognize faces at a distance above 1 meter. Most AMD individuals (91%) report looking eccentrically at faces during the identification process; only a single individual (9%) reported looking directly at the face. On the contrary, most RP individuals (79%) report looking directly at the face, while few (21%) report looking eccentrically.

4.3 ESGAD questionnaire

The results of this questionnaire highlight the daily functional and social difficulties encountered by the considered population: reading and writing (functional) and face recognition & emotion recognition (social). The main expectations of the distinct groups for assistive smart glasses devices are presented in Fig. 5. The studied population reports the following difficulties with daily functional tasks: reading (94%), writing (84%), localization of objects (60%), mobility (56%), identification of objects (50%) and meal preparation (30%). The participants report the following difficulties with daily social tasks: face identification (94%), Emotion Recognition (70%), other’s empathy (68%), establish contact with others (22%), sports (12%) and cultural activities (10%). Surprisingly, less than a third of the studied population (32%) is aware of the existence of smart glasses and their potential as an assistive device for visually impaired individuals. In their daily life, they currently rely on the following assistive devices: white cane (88%), magnifying glass (66%), digital screens for reading (42%), guide dog (4%) and trustworthy persons (4%). They report the following expectations concerning the functionalities that would be useful for them on assistive smart glasses: face recognition (88%), reading (86%), mobility (48%), object localization (32%), emotion recognition (32%), object identification (30%) and GPS functions (28%). Assuming they would have such device with face recognition functionality, they would like to obtain the name of a target person (100%) and to distinguish strangers from known people (54%). They all rejected the idea of having the information about age. Finally, most participants reported the aesthetic (84%) and the comfort (100%) as being important factors.

Fig. 5
figure 5

Expectations on the functionalities that an assistive smart glasses device should provide. Items are ordered by preference of the whole population and the results are presented for each visual pathology

Full size image

As visible in Fig. 5, most groups share the same expectations: Face recognition and Reading. The ONL group reports fewer daily difficulties than others in general and the only functionality required by the majority is reading assistance. Compared to other groups, Object localization and identification seemed to be particularly important for RP group.

5 Discussion

The analysis of the results obtained from the three different questionnaires provides insights on the general quality of life, daily difficulties, needs and expectations of the studied populations. The results notably highlight the difficulties with social interactions and the needs for an assistive technology designed for visually impaired that focuses on facilitating daily interactions with others. The analysis of the results of the NEI questionnaire shows that the studied populations are largely impacted in their quality of life by the symptoms of their pathology. Their overall score, on average, is low and the comparison of the subscale scores obtained in this study with the score obtained for the standard population presented in the original study support that claim [23]. This effect might potentially originate from the sample population considered in our study: the participants were recruited through the SBV association, which often consists of persons who contact them because they suffer from severe symptoms. The analysis also highlights the difficulties experienced daily for social interactions; this item is frequently reported as a problem and considered as a strong discomfort by most participants. The specific items related to face recognition outline the strong impact of distance, indicating that persons closer to one meter would not always need to be recognized by the system. The degree of familiarity of a face only plays a minor effect and should not necessarily be taken into considerations (i.e. they have the same difficulty recognizing the face of a very close-relative or the face of an occasional colleague). The importance and negative impact of luminosity and contrast are also clearly highlighted by the results as key elements to take into consideration; notably in terms of the information provided through the display of the glasses, similarly to the recent work performed by Zhao et al. [39].

Table 2 Preferred functionalities expected by most of the population within each group (bold: ≥ 50%, light-grey: ≥ 30%)
Full size table

The analysis of the DRFE questionnaire characterizes the strong difficulties with daily social interactions for the studied population. The most problematic items are face identification, visual attention in groups and -to a lesser extent- emotion recognition. The difficulty to identify faces is reported by almost all participants and types of pathologies as a strong or very strong discomfort. Only few individuals of the ONL group, presenting lighter visual deficit symptoms, did not report strong discomfort for that item. Almost all participants (98%) reported difficulties to identify faces at distances superior to one meter. A majority of participants (68%) reported difficulties at distances inferior to one meter. Inspecting the results at the pathology level, it appears that more individuals from AMD and RP groups reported having difficulties at close range compared to other groups. This supports the idea that face identification assistance at close range must be tailored according to each individual’s preferences. Difficulties with visual attention, i.e. to understand when one or several interlocutors are looking at the VI individual, are reported as occurring frequently to very frequently for most of the studied population. Such difficulties can notably be worsened in group conversations where turn-taking is often complex and implicitly managed through eye contacts (mutual-break and mutual-hold). According to the results, individuals from the AMD and ONL groups are less impacted by this difficulty. It is interesting to note that visual attention in live conversations has only been addressed in a few studies and it seems that real expectations from VIs have not sufficiently been considered leading to prototypes that were not well accepted by the target population [26]. Understanding facial expressions of emotion was reported as a moderate discomfort on average, however the heterogeneity of the reported discomfort amongst the studied population emphasizes the need to provide the option to tailor that functionality for each individual. Many participants reported relying on their auditory sense to infer the emotion of their interlocutor, which might not always possible in everyday life situations. Note that the ONL and Others populations reported very heterogeneous discomfort for that specific item, showing strong variability even within groups.

The analysis of the ESGAD questionnaire highlights the fact that expectations for the smart glasses are largely shared across all groups considered in this study. The results presented in Table 2 highlight the different functionalities expected and their preferred order for each group. The numbers, in bold and in light grey, represent expectations desired by more than 50% and more than 30%, respectively. All considered pathologies share the same top-2 expectations. Face recognition was, by far, the most requested functionality, which clearly indicates that participants need additional aids to support such daily social function. Reading was the second most desired functionality, which could indicate that they may not be fully satisfied with their current text reading aids. For face recognition, participants reported that they wish to be notified not only about identity of known persons but also about whether or not that person is known to them; a feature that is often absent in most existing systems. Mobility and object localization/identification functionalities were also requested by many participants. Surprisingly, although many individuals reported frequent difficulties with visual attention detection in the DRFE questionnaire, it was not requested as a desired functionality. This may be explained by the fact that “Visual attention detection” was not part of the proposed items in the ESGAD questionnaire; participants had to mention it explicitly in the “Others” section. Facial emotion recognition was only requested by 33% of the population, which is in par with the moderate difficulties reported for that specific item. Interestingly, the RP group seems to expect all functionalities, although their composite and general vision scores were not significantly lower in the NEI questionnaire. This demonstrates the importance of the two specific questionnaires developed in this study to better assess the real needs and expectations of considered populations.

Finally, the current study has some limitations that are resumed below. The information considered for the visual acuity was self-reported by the patients; indeed, most patients were not aware of their exact medical visual acuity, which did not allow assessing the correlation between visual acuity and the other factors considered in the study. The sample population considered was composed of only few young persons and mostly of female individuals, sharing a common cultural background as all individuals were living in the french region of Switzerland. These specificities might have impacted the results; the age (M = 61.68, SD = 14.14) might notably have impacted their knowledge and attitude towards adopting new technologies, while the shared regional background might have an impact on their expressed needs. The limited size of the ONL and Gl groups and strong heterogeneity of the ONL and Others groups may have hindered the possibility to obtain additional statistically significant results. For all these considerations, the generalizability of the findings to a worldwide population of visually impaired must be taken with caution. It would be interesting to reproduce the same study in different regions of the world to reduce the regional impact and increase the size of the population samples. Furthermore, the current study could not address several important topics that should be investigated in future studies; an in-depth analysis of the preferred feedback channels of the considered populations should notably be further investigated as it represents a crucial aspect for future developments. The economical aspect of smart glasses device was not considered, although it would be interesting to know the acceptable price-range users would be willing to pay. The privacy issues, in terms of public acceptance and legal consideration should also be further investigated, notably for specific public contexts such as classrooms. A recent study already demonstrated that people tend to be more willing to grant access to their image to help people with disabilities [1]. Reported as a main drawback of existing solutions in the literature [35], future developed solutions should focus on instructions, accessibility and support to assist the users with the initial setup and daily use of the proposed systems.

As mentioned in the state-of-the-art discussion, most research articles that investigated the needs of visually impaired individuals in terms of technological assistive devices focused on specific context such as stigmatization [30, 32] or on specific types of applications such as shopping [2] or mobility [40]. Most of those studies have been performed through open interviews and/or custom questionnaires with a limited population [29]. Furthermore, considered populations do not represent the whole continuum of potential pathologies in visual impairment conditions and the subjects often consist of persons in the academic world. In this work, we tried to provide an overview of the general needs of visually impaired individuals, while covering several distinct pathologies in order to provide a better understanding of the common and distinct needs and expectations. We used a combination of standard and custom questionnaires in order to provide several different perspectives on the topic: general health conditions, daily difficulties, daily social difficulties and specific questions regarding the potential of smart glasses as an assistive device. Finally, we relied only on voluntary subjects that are registered to the Swiss Association for the Blind and Visually Impaired and who have been interviewed with a well-defined methodology.

6 Summary of findings

We summarize in the following the most important findings that could be inferred from this study in order to guide the development of future augmented reality researches, software and products targeting visually impaired individuals. These findings have been divided into two distinct groups: objective findings extracted from the quantitative data and subjective findings extrapolated from discussions with the participants.

From the analysis of the questionnaires’ quantitative data, the following findings can be highlighted:

  1. 1.

    Independently of their pathology, VI individuals request face detection/recognition and text reading assistance functionalities. In addition, the current study clearly highlighted the desire of VI to have those functionalities regrouped on a common device.

  2. 2.

    Face recognition must be tailored according to the pathology and to each individual:

    • All considered pathologies experience difficulties recognizing faces at a distance superior to one meter;

    • Face recognition at close distance must be tailored for each individual according to its preferences, there is no common agreement amongst pathologies;

    • ONL expressed less interest in face recognition functionalities;

    • RP and Gl required face detection for peripheral vision mainly (close and long range).

  3. 3.

    Face recognition should not be customized according to familiarity of the person to recognize. Indeed participants reported the same difficulties whether trying to recognize very familiar or less familiar persons.

  4. 4.

    Visual attention detection (detecting if an interlocutor is looking at them) in face-to-face and group discussions is problematic for VI individuals; it could be supported by smart glasses devices.

  5. 5.

    Recognition of facial expressions of emotion is not a functionality requested by the majority of visually impaired individuals.

  6. 6.

    Participants reported the importance of design and comfort of the assistive device.

  7. 7.

    The device should be able to provide physical means to limit strong luminosity changes (ex. possibility to use physical shades): the RP, AMD and Gl populations reported being particularly affected by strong luminosity changes.

From the discussions and open questions from the questionnaire, the following aspects can be mentioned:

  1. 1.

    Smart glass devices are seen by most VI as a very promising technology for two main reasons, namely hands-free interaction and information contextualized according to head-gaze direction (head pointing direction).

  2. 2.

    Future developed systems should ensure the provided feedback does not hinder the remaining senses of the user when unsolicited; i.e. frequent and long audio notifications should be avoided whenever possible. Audio channel is still considered by many individuals as the optimal channel to convey large quantity of information.

  3. 3.

    Adaptive display of visual information is crucial to ensure sufficient contrast (i.e. adaptive color palettes based on context). Additional studies and experiments will need to be performed in this domain.

  4. 4.

    The design of a device should avoid stigmatizing the user or making him stand-out. This aspect seemed particularly important for the participants and echoes point (6) mentioned above.

  5. 5.

    It is important to provide the possibility to wear standard glasses underneath the assistive device or use prescription lenses.

7 Conclusion

The current work provides, to our knowledge, the largest study focusing on needs and expectations of visually impaired individuals for assistive smart glasses. The outcomes of the study are a set of questionnaires that provide the possibility to assess the quality of life, functional difficulties and needs as well as technological expectations of VIs and a set of findings highlighted from the analysis of the questionnaire which should help guide future researches in the field. The study, covering interviews with 50 participants having various VI pathologies, provides a detailed view of the typical difficulties encountered by the different participants and their needs. The analysis of the results provides insights on the needs and expectations according to the distinct pathologies, highlighting the need of tailored or highly customizable solutions for specific items. The set of findings, inferred from the analysis of the results and the interviews resumes the most important elements that must be addressed to successfully develop solutions for assistive smart glasses that will ideally support VIs in their daily life. In the future, concrete solutions for smart glasses will be developed based on the main findings of the study; we notably plan to develop solutions specifically adapted to support social functions such as face identification, visual attention detection and recognition of facial expressions of emotion. A second important future step is to extend the proposed questionnaire to cover feedback aspects. The types of feedback, their modulation and robustness to errors are key elements to consider for a successful acceptation by visually impaired individuals.