Abstract
Human–computer interaction (HCI) is particularly interesting because full-immersive technology may be approached differently by users, depending on the complexity of the interaction, users’ personality traits, and their motivational systems inclination. Therefore, this study investigated the relationship between psychological factors and attention towards specific tech-interactions in a smart home system (SHS). The relation between personal psychological traits and eye-tracking metrics is investigated through self-report measures [locus of control (LoC), user experience (UX), behavioral inhibition system (BIS) and behavioral activation system (BAS)] and a wearable and wireless near-infrared illumination based eye-tracking system applied to an Italian sample (n = 19). Participants were asked to activate and interact with five different tech-interaction areas with different levels of complexity (entrance, kitchen, living room, bathroom, and bedroom) in a smart home system (SHS), while their eye-gaze behavior was recorded. Data showed significant differences between a simpler interaction (entrance) and a more complex one (living room), in terms of number of fixation. Moreover, slower time to first fixation in a multifaceted interaction (bathroom), compared to simpler ones (kitchen and living room) was found. Additionally, in two interaction conditions (living room and bathroom), negative correlations were found between external LoC and fixation count, and between BAS reward responsiveness scores and fixation duration. Findings led to the identification of a two-way process, where both the complexity of the tech-interaction and subjects’ personality traits are important impacting factors on the user’s visual exploration behavior. This research contributes to understand the user responsiveness adding first insights that may help to create more human-centered technology.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Human–computer interaction (HCI) can be seen as a consequence of the widespread usage of personal computers by non-professionals with close to zero computer knowledge. Ubiquitous computing is then the direct response to this mass adoption and, of course, to the advent of internet popularity, which boosted the spectrum of possible applications.
This evolution led to several implementations in different contexts: factories, health, and home environment. Home automation, also referred to as smart home systems (SHS), domotics, or intelligent environment, is a multidisciplinary field of research, a subset of ubiquitous computing, aiming at improving the users’ quality of life inside and through the habitation, by using engineering, informatics, user experience and architecture (Navarro-Tuch et al. 2019). The adjective “smart” is associated with these systems because of their capacity to meet complex needs with a certain degree of automation (Kanemura et al. 2013), and additionally adapting and overcoming issues in heterogeneous contingencies. SHS is also typified by easiness to configure the system, with reduced effort and cost compared to a normal habitation (Corno and Razzak 2014), and by multifunctionality of the single devices (Mekuria et al. 2019).
Usability studies in SHS are generally conducted through the implementation of qualitative methods, such as in-depth interviews and evaluative scales, focusing on psychosocial attributes involving relational and functional aspects like routines, representations, and meanings (Gram-hanssen and Darby 2018). Individuals’ subjective experience is now the focal point of much research, aiming at studying the well-being of the user and creating human-centered systems (Lopez-Aguila et al. 2020). Psychological factors, such as personality traits and behavioral variables, are consequently considered to be dense of useful information regarding usability and users’ experience inside SHS, and for this reason, they are frequently included in research design protocols. Indeed, the relation between personality and cognitive factors on interface interaction was already studied by previous work (Green and Fisher 2012), where individual differences and interactive visualization are key-elements in a computer-mediated activity. Also, between the others, it was found that locus of control (LoC) is an influential personality trait for every interface interaction. For example, they found that people with more internal LoC, so those who believe in personal control over life event, tend to interact more easily with tech-interfaces (Green and Fisher 2012). Other authors also found that people with a more internal LoC will have a generally more positive attitude towards technology (Arndt et al. 1983). However, so far there is a lack of information about the relation between LoC and the tendency to interact with technology.
Another interesting contribution to user experience (UX) comes by considering the motivational systems adopted by users, that is their differences in subjectively approaching either avoiding interactions with technology and SHS. Behavioral inhibition system (BIS) and behavioral activation system (BAS) (Gray 2003) were designed to measure two motivational systems, as an explicit indicator of individual differences in the motivational significance (Balconi and Mazza 2009). These systems showed to correspond to stable personality traits (Cloninger 1986; Depue and Collins 1999). Specifically, BIS is connected to negative implications of general behavior, and higher levels of it may be interpreted as an experience of negative feelings about a new situation, with the subject’s propensity to avoid it (Gray 1994). Higher BIS activation is associated with inhibition, in the sense of abrogation of behavior, vigilance, anxiety, and very strong BIS measures that may correspond to anxiety-related disorders (Quay 1988). Instead, BAS is generally positively affected by pleasant experience (Gray 1994), with perhaps a user’s tendency to approach the artifact within this context. Specifically, BAS reward responsiveness (BAS-RR) is very sensitive to signals of reward and non-punishment and is highly correlated to the feeling of optimism and happiness (Gable et al. 2000), and therefore is a helpful metric when a subject is facing new environments. Both these two systems are reliable, also because they have been correlated before to specific neural oscillations (Balconi et al. 2009) and helpful for exploring and understanding the perception of individuals on a new experience (Balconi and Mazza 2010) and therefore their application to these new technology environments can be valuable.
Within this complex context, self-report methods are needed but not sufficient; indeed, their major limitation is that they provide only the subject’s explicit and conscious information, leaving out information about the implicit, automatic and unconscious dimension of attention and behavior. Taken this into consideration and in line with the definition of usability evaluation, identified as a set of techniques where users interact with an artifact, while behavioral data are collected (Dumas and Redish 1993; Wichansky 2000), the eye-tracking method is considered an efficient approach, bringing reliable behavioral metrics (Merwin 2002). Because of its capacity to provide data on processes which are not available to introspection or rather under the surface of conscious awareness, eye-tracking was adopted for studying user behavior (Chen and Pu 2010), user cognitive traits (Tsianos et al. 2009) and to indirectly infer on attentional processes (Ooms et al. 2012).
Related to technology adoption, it was applied to study how users interact with commands and menu interface (Altonen et al. 1998; Byrne et al. 1999), product usability in general (Cowen et al. 2002), and specifically HCI (Hutchinson et al. 1989; Levine 1984). Moreover, after the adoption of wireless technology, eye-tracking tools are considered highly valuable in applied contexts (Mihajlović et al. 2014), offering an optimal compromise between ecological and internal validity with real-time data on the attentional visual system (Liu and Heynderickx 2011).
Regarding the metrics, the concept of fixation, defined as a period between two saccades, of about 180–330 ms, where the eye is relatively immobile and where visual information processing is assumed to take place (Latour 1962; Volkman 1976), is used to derive several indices. Generally, in studies concerning HCI, the common metrics to evaluate a peculiar areas of interest (AOI) are: number of fixation (Backs and Walrath 1992; Cowen et al. 2002; Wee et al. 2020), defined as the amount of fixation in a certain time frame (Lykins et al. 2006), fixation duration (Cowen et al. 2002; Bialkova et al. 2020) described as the average time duration of certain fixations in a time frame, and time to first fixation (Buscher et al. 2009; Bruder and Hasse 2020), which represents the time spent from stimuli onset to the first fixation arrival (Lai et al. 2013).
By crossing both self-report and eye-tracking methods a full understanding of UX in SHS can be elicited, adding to subjective perception reliable behavioral metrics. To our knowledge, although the use of eye-tracking does not represent an innovation for the evaluation of the UX, the novel aspect consists in its application in a SHS context: in fact, no studies before have ever investigated visual system behavior in domotics environment, and, besides, no correlations between the psychological constructs such as LoC and BIS/BAS system mentioned above and these metrics have been explored.
Therefore, this research aims to observe possible differences in users’ visual behavior in heterogeneous tech-interactions in a SHS, by considering the number of fixations, fixation duration, and time to first fixation metrics and additionally taking into account possible relations with individuals’ psychological attributes such as LoC (external/internal) (Rotter 1966; Arndt et al. 1983), personality traits, such as BIS/BAS (Gray 2003) and the construct of user experience (Hekkert 2006).
Based on our hypotheses, higher complexity in tech-interactions may have a direct impact on visual behavior in the subjects. Firstly, we expected to observe a higher number of fixation, generally used as indices of the ease of processing (Hyönä et al. 1995) in simple tech-interactions compared to less complex tech-interaction.
Secondly, we also hypothesize to collect longer fixation durations in complex tech-interaction, as a direct effect of the difficulty of the foveal task (Jacobs 1986; Hooge and Erkelens 1996) and time to first fixation in complex tech-interaction, as a reflection of high cognitive workload and higher cognitive effort (Shojaeizade et al. 2016).
Thirdly, during the interactions between SHS and users, we expected to observe a general direct relationship between LoC and number of fixations because previous studies showed that individuals with internal LOC tend to interact more easily with tech-interfaces (Green and Fisher 2012). Specifically, we expected a positive relationship between internal LoC and number of fixations, and a negative one between external LoC and number of fixations.
Fourthly, since psychological traits and subjective dimensions were found to have a direct impact in visual processing (Toker et al. 2012), working memory (Turner and Engle 1989) and in general performance, we hypothesized the existence of linear relations between visual metrics and BIS/BAS scales. Specifically, we postulate that higher BIS scores are negatively correlated to the number of fixations because of a bigger tendency to avoid new experiences because of a personal inner sense of discomfort and general uneasiness. Moreover, we expect to detect a negative correlation between BAS, especially for the subscale reward responsiveness (BAS-RR) scores and fixation duration, caused by a more impulsive and faster search in the environment during the tech-interaction by subjects with BAS-RR compared to a more conservative and slow exploration from people with BIS (Balconi et al. 2011).
2 Methods
2.1 Participants
Nineteen healthy subjects (Mage = 25.05, SDage = 3.05, Age Range: 18–27, nmale = 7) were recruited for the study. Criteria for inclusion were normal, or corrected to normal, visual acuity. Exclusion criteria were the previous knowledge of the domotics show loft, the presence of sensory and cognitive deficits, a history of psychiatric or neurological diseases, and the ongoing concurrent therapies based on psychoactive drugs that can alter central nervous system functioning. No compensation was provided for their participation in the study. All participants voluntarily took part in the experiment and gave informed written consent for participating in the study. The research was conducted following the principles and guidelines of the Helsinki Declaration and was approved by the Ethical Committee institution of the Department of Psychology of Catholic University of the Sacred Heart, Italy.
2.2 Procedure
The experiment took place in a home automation environment in Milan (Italy), a showroom owned by tech company Duemmegi S.r.l., developer and seller of domotics systems. Participants were asked to fill in pre-experiment questionnaires, such as BIS/BAS scales (Leone et al. 2002) and the LoC questionnaire (Craig et al. 1984). After the montage of wireless eye-tracker (Tobii Eye-tracker PRO 2), a researcher introduced the subject to the experimental setup. Wireless technology guaranteed no physical restrictions. Subjects were asked to interact and activate five different tech-interactions in the SHS. Every interaction would lead to different outcomes, induced by the domotics. All points of interaction were situated in different parts of the house. Each of these interactions was activated by a smartphone app. Both the app and device were provided by the research team followed by a standard task explanation where participants familiarized themselves with the app interface.
The points of interaction were five: hall, kitchen, living room, bathroom, and bedroom. All interactions were started through the use of the app. The subject was only required to activate the tech-interaction through the app and observe the environmental effects of its action. Every tech-interactions had a certain amount of complexity, based on the response provided by the SHS. During the experiment the SHS responses became more and more sophisticated and complex, involving and connecting more devices together. In the hall, the interaction consisted of activating a light. In the kitchen condition, the activation of the command would provoke the transformation of an apparently normal set of desks in a fully accessorized kitchen with stoves and oven. Moreover, the living room interaction consisted of activating a multimedia projector on a screen appearing on the wall. The fourth one (bathroom) involved the presentation of some features like specific lights for make-up or shaving sessions, chromotherapy, and turning on a smart device. Lastly, the bedroom area consisted of a full home bedtime mode activation. Blinds were closed, the home was secured by locking external doors, and gas and lights were turned off. The total exploration lasted approximately 30 min. Users were asked to pay attention to the experience provoked by the activation and to focus on the interaction with the SHS. Finally, the user experience questionnaire was administered.
2.3 Eye-tracking measures
For this study, a Tobii Pro Glasses 2 (Version 1.95, 07/2018. Tobii Pro AB, Stockholm, Sweden) was used, composed by the glasses connected to the recording unit (Fig. 1).
Near-infrared illumination is used to create and register reflection patterns on the Pupil Centre and Cornea of the subjects’ eye (PCCR). The tool, equipped with a gyroscope and accelerometer has a wide-angle scene camera (1920 × 1080 video resolution), with four infrared sensors. An eye tracker is able to capture 3D real-time information through fixations and saccades which are then analyzed via Tobii Pro Glasses Analyzer, to derive the viewer’s attention patterns. Areas of interest (AOI), defined as “area of a display or visual environment that is of interest to the research or design team” (Jacob and Karn 2003), were then created for each tech-interaction, with specific attention to take in consideration only the parts of visual fields actually involved during the SHS interaction.
In eye-tracking studies, several different parameters may be taken into consideration to study the type of ongoing attentional processing (e.g. Hyönä and Nurminen 2006). Fixation metrics have been used as an indicator of cognitive effort in many fields (Doherty et al. 2010), like UX, clinical, marketing studies. In this study, we considered metrics based on fixation, defined as “eye movements which stabilize the retina over a stationary object of interest” (Duchowski 2003), such as number of fixations, time to first AOI fixation, and fixation duration. All the data are referred to a standard time frame after the activation via smartphone of the tech-interaction, while the SHS was engaging in some actions.
2.4 Locus of control
To evaluate the LoC, we administered the locus of control behaviour questionnaire (LCB) (Craig et al. 1984). LCB is designed as a tool the evaluate the LoC in subjects in different situations. The questionnaire is composed of 17 items, with a 5-point Likert scale. Seven of the 17 items are referred to internal control, the other ten to the external one. For this study, we used the LCB Italian version and we calculated internal and external LCB corresponding respectively to an individual’s tendency to believe that he can control events in his life, the future, and their outcomes by his efforts, and, for external LCB, the propensity to attribute importance to outside forces (Farma and Cortinovis 2000). Both the scores for the internal and external scales are calculated and here reported: external (M = 20.21 SD = 6.54), internal LCB (M = 10.15 SD = 2.59).
2.5 BIS/BAS scales
To evaluate the two motivational systems, the Italian version of BIS/BAS scales was adopted (Carver and White 1994, Leone et al. 2002). This questionnaire, including 24 items (20 score-items and four fillers), is composed of 5-point Likert scale items, is used to understand the subject’s propensity to experience new situations or artefacts. As mentioned above, BIS (seven items) is sensitive to signals of punishment and nonrewarded, therefore its activation may cause a state of inhibition of movement toward goals. Instead, BAS (item 13) activation may be caused by the experience of positive feelings (Gray 1990), greater BAS sensitivity is connected to a person’s tendency to engage in goal-directed efforts. The BAS items are referred to three subscales: BAS drive (four items), the motivation to achieve a certain goal, the BAS reward responsiveness (five items), that measures the impact on the subject of environmental rewards, and finally the BAS fun seeking (four items), corresponding to the natural personal motivation to find and engage with new rewards.
Based on these measures, the scores for BIS and the three BAS subscales were calculated. The mean values and standard deviations are reported for BIS (M = 27.21 SD = 3.92), for reward (M = 19.68 SD = 2.66), for drive (M = 13.32 SD = 1.76), and for fun seeking (M = 13.95 SD = 2.93).
2.6 User experience questionnaire
In order to evaluate UX, the user experience questionnaire was adopted (Laugwitz et al. 2008; Capellini et al. 2015) and administered to subjects after completing the experimental phase. The questionnaire is composed of six scales (attractiveness, perspicuity, dependability, efficiency, novelty, and stimulation) and a total of 26 items, using a 7-point Likert scale. Based on these measures, the 6 scales mean and standard deviation values are calculated and reported: attractiveness (M = 2.18 SD = 0.58), perspicuity (M = 2.55 SD = 0.47), dependability (M = 1.13 SD = 0.62), efficiency (M = 1.76 SD = 0.61), novelty (M = 1.88 SD = 0.70), and stimulation (M = 1.67 SD = 0.64).
2.7 Data analysis
For eye-tracking data, a set of three repeated measures ANOVA was conducted. Each analysis was separately applied to the eye-tracking dependent measure (number of fixation, fixation duration and time to first fixation) to determine whether there was a statistically significant difference during the five tech-interaction conditions (entrance, kitchen, living room, bathroom and bedroom) for the three considered metrics. For all ANOVA, degrees of freedom were corrected by Greenhouse–Geisser epsilon when appropriate. Post-hoc analysis (contrast analysis with Bonferroni corrections for multiple comparisons) was applied. The size of statistically significant effects has been estimated and reported via eta squared (η2) indices. Secondly, Pearson’s correlation coefficients, with Bonferroni corrections for multiple comparisons, between eye-tracking metrics and questionnaire scores (LoC, BIS/BAS, and UX scales) were calculated.
3 Results
3.1 Eye-tracker
Number of fixations: As shown by ANOVA, a main effect for tech-interaction was found (F[4, 72] = 4.740 p = 0.004, ƞ2 = 0.345). Pairwise comparisons revealed a significant (p ≤ 0.002) higher number of fixations in the entrance interaction (M = 62.17) compared to the living room one (M = 35.52) (Fig. 2a). No other significant effects were found.
Fixation duration: From ANOVA no significant difference between the means was observed (p > 0.05).
Time to first fixation: As shown by ANOVA, a main effect for tech-interaction was found (F[4, 72] = 5.36 p = 0.002, ƞ2 = 0.437). Pairwise comparisons revealed significant (p ≤ 0.005) higher time to first fixation in the bathroom (M = 5.710) compared to the kitchen (M = 1.180) and a significant difference (p ≤ 0.005) between bathroom and living room interaction (M = 1.211) (Fig. 2b). No other significant effects were found.
3.2 Pearson correlation coefficients
Pearson’s correlation coefficients between eye-tracking metrics and questionnaire scores were applied to assess the relationship between psychological characteristics and visual system behaviour, no outliers were present. Only significant results are reported.
LCBQ: A statistically significant negative correlation was found between the number of fixations in the living room interaction and external LCBQ [r(17) = − 0.613, p < 0.045]. Moreover, another significant negative correlation was found between the number of fixations and external LCBQ the bathroom condition [r(17) = − 0.688, p < 0.019]. Instead, no significant correlations were found for internal LCBQ and time to first fixation and fixation duration.
BIS/BAS scales: A statistically significant negative correlation was found between the number of fixations in the hall interaction and BIS score [r(17) = − 0.694, p < 0.038]. Regarding BAS reward responsiveness scale, a significant negative correlation was found between this measure and fixation duration for both bathroom [r(17) = − 0.658, p < 0.028] and living room [r(17) = − 0.681, p < 0.021] conditions. No significant correlation was found for the BIS scale with fixation duration and time to first fixation, and for the BAS scale with number of fixations and time to first fixation.
4 Discussion
The study provided information on visual system behaviour and psychological factors using the eye-tracking system and self-report measures. More specifically, participants were asked to interact in a SHS environment, composed of five tech-interactions (hall, kitchen, living room, bathroom, and bedroom), with different levels of complexity, while visual behavioural was collected. Additionally, self-report measures related to individual differences (i.e. motivational systems for approach and avoidance behaviour), locus of control, and users’ experience were collected to obtain useful information on users’ subjective information for interpreting SHS interaction. A possible interpretation of the main results of this experimental study is reported below.
Firstly, concerning the number of fixations, results from ANOVA partially confirmed the first hypothesis. In fact, data showed a significant higher number of fixation in the entrance condition compared to the living room, showing easiness to process and elaborate in terms of higher number of fixations (Hyönä et al. 1995) when facing a simple tech-interaction (entrance), compared to a more composite one (living room), where the subject had a less visual explorative behaviour. The number of fixation can be seen as a direct function of visual exploration behaviour (Glöckner and Herbold 2011), and a possible interpretation is that subjects put more attention in visually explore, observe and engage with the SHS when the domotics outcomes were simple and not too much cognitively complex and challenging. This result also macroscopically confirms the need for a natural interface always more human-centred and easy to use and approach for users.
Secondly, regarding time to first fixation results, we observed a significantly higher time duration in the bathroom interaction compared to both the kitchen and living room ones. These findings can be interpreted in coherence with the number of fixation results: indeed, also in this situation, a more multi-faced tech-interaction (bathroom) required more time to be fixated for the first time, compared to simple ones (kitchen and living room). Time to first fixation can be interpreted as a tendency of the user to engage and a level of motivation to detect important and salient information in an interface (Buscher et al. 2009). Therefore, data showed that a high-complex reaction from the SHS, such as the bathroom with chromotherapy multisensory options, can indirectly make users not focus on the tech-interactions, displaying slower visual responses. Future studies might deepen this complex interactive dynamic.
Thirdly, regarding LCBQ, two strong negative correlations were found, between the number of fixations and external LCBQ during both the living room and bathroom tech-interactions. We failed to detect this tendency in the other conditions (kitchen, hall, and bedroom) and for the internal LoC, as we expected. Even so, this result can be interpreted as a tendency of individuals with low external LoC, so those who are inclined to attribute small importance to external and unpredictable factors, to explore more actively complex tech-interactions thus showing high interest in them. Indeed, visual search behaviour is highly dependent on the number of fixations (Togami 1984). The principle assumption behind this is the eye-mind hypothesis (Ball et al. 2006), which states that when a subject is fixating an artefact, the brain system is engaged in cognitive processing, also reflecting the level of interest towards that stimulus. The other way around, if subjects have higher external LoC they may tend to explore less and be less intensively visually engaged with SHS. Results partially confirmed Arndt et al. (1983) evidence on the impact of LoC on the interface interaction, previously suggesting that internally controlled individuals were more curious and less anxious towards technology artefact.
Lastly, concerning BIS/BAS scales, a significant solid negative correlation was found between the BIS score and fixation number in the Entrance condition. These findings could be indirectly interpreted as a tendency of avoiding new situations (Gray 1994). BIS may be explained as a predisposition of the subjects to prevent themselves from anxiety-provoking conditions (Fowles 1987; Carver and White 1994), with reduced visual exploration behaviour. Therefore, in a domotics context, it could be possible that people with a major tendency to be inhibited (higher BIS score) could repress the exploration behaviour of a less-known complex environment. However, this is the first time BIS was explored in relation to eye-gaze data in SHS; hence, further neuroscientific evidence supporting our explanation is necessary.
Instead, concerning BAS, a correlation between reward responsiveness scale scores and fixation duration was found in two tech interactions (living room and bathroom). Even if we failed to detect significant correlations between BAS scores and fixation count, this data, as opposed to BIS result, may indicate a tendency from subjects with higher BAS reward responsiveness scores to explore more the point of interaction, with a reduced fixation duration average. In fact, contrary to what may seem logical, no robust relationship between fixation duration and data processing was ever demonstrated (Salthouse and Ellis 1980). According to Henderson and Hollingworth (1999), an impacting factor on time length for fixations may be the type of mental process generated by the person: it seems that fixation durations are longer during scene memorization compared to search ones. A possible explanation may be that people with a bigger tendency to approach the new situation in search of rewards (higher BAS reward responsiveness scores) may tend to have more visual exploration type of experience, which is more impulsive, fast and emotional, reducing the average time per fixation. Also, it was found that longer fixations are related to complexity (Salthouse and Ellis 1980; Hooge and Erkelens 1998) and it may be that people with lower BAS RR tend to find it more difficult to interact with SHS, provoking longer fixations. Future studies might unveil stronger relationships between individual differences (personality traits) and eye-gaze behaviour.
Linking both significant results from the experimental condition and cognitive psychological factors correlations, it can be said that both complexity of the tech-interaction and personality traits might have a relevant impact on subjects’ experience and interface with a SHS.
Firstly it seems that the level of complexity of HCI may have an impact on both number of fixations and time to first fixation, reducing the first and increasing the second, for the situation’s inner nature, with a reduced visual exploration of the users, and secondly, we found that people’s characteristics may have an impact over the exploration, specifically, we found that people more sensitive to discomfort caused by the new situation (higher BIS scores), tend to visually avoid the SHS tech-interactions (lower number of fixations), interpreted as an escaping strategy and that subjects who give importance to environmental reward (higher BAS reward responsiveness scores) tend to explore the setting and the interaction faster and more impulsively (reduced fixation duration). Even if we believe future studies should detect this pattern of observations in all the tech-interaction conditions, from these first insights a two-way process can be inferred in subject visual behaviour in SHS: a top-down process, caused by psychological and motivational traits, which may impact the user visual behaviour inducing a specific predisposition when interacting with a domotics, and a bottom-up process that corresponds to the level of complexity of the HCI that showed to have an impact on visual behaviour. Both these two factors may contribute to encounter different experience and subjective feelings in users and therefore model and shape the interaction with SHS.
To conclude, the present research investigated the relationship between personal psychological construct and trait in five different tech-interactions in a SHS, in order to piece together human and machine and more specifically personality, motivational systems, and interaction with smart devices. Results should be strongly confirmed by future studies, possibly using more sophisticated statistical models; however, they may be useful to design better natural user interface. The study provides new interesting insights, although, for both academic and professional purposes. Firstly, new directions of research can be taken, to confirm our first results and also take into consideration other processes such as memory or multisensorial experience. Secondly, architects, designers, and engineers, to provide optimal comfort, may maximize users’ delight, by considering personal characteristics and using them to set the smart devices around the subject, fully adopting the human-centred philosophy. Moreover, two main conclusions can be drawn for the design of the SHS: first, there could be a need for a natural interface always more human-centred and easy to use and approach for users, as suggested by the first result; secondly, according to our second evidence, a too complex interactive dynamic in SHS environments can slow down user’s visual response.
Significant results were found remarking both the influence on HCI of psychological variables and technical aspects that refer to the level of complexity of the tech-interaction. Strengths of the research can be identified in the use of a real and ecological SHS which guaranteed a good external validity, where the subject could experience a full immersion in a real tech-environment, and at the same time the full possibility, due to SHS owner’s availability, to protect and isolate the setting from external factors. Another positive aspect regards the study’s innovation where the complexity of tech-interactions and motivation systems variables are studied together with visual system metrics.
Despite what just pointed out, the present study is not without limitations. Firstly, our hypotheses were confirmed only in two (living room and bathroom) of the five tech-interaction conditions. We assume that an ampler sample size could conclusively clarify a general effect of the examined psychological variables on visual system behaviour. Furthermore, the ecological strength point of the study design implicated a low level of standardization in the condition (different from a laboratory setting). Also, the analysis carried out only adopted single eye-tracking measures, not considering specific gaze dynamics and patterns or innovative approaches but standard statistical models.
Lastly, the study did not take into consideration variables such as gender and age in extensive analyses, which are known to be impactful in HCI. Future research could consider other eye-tracking metrics also based on saccades and investigate possible differences between various interface systems, for example, vocal or touch-screen ones, observing the psychological (both cognitive and emotional) predisposition that may create preferences over it.
Data availability
The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.
References
Altonen A, Hyrskykari A, Räihä K (1998) 101 Spots, or how do users read menus? In: CHI 98 Hum Fact Comput Syst. https://doi.org/10.1145/274644.274664
Arndt S, Feltes J, Hanak J (1983) Secretarial attitudes towards word processors as a function of familiarity and locus of control. Behav Inf Technol 2:17–22. https://doi.org/10.1080/01449298308914466
Backs RW, Walrath LC (1992) Eye movement and pupillary response indices of mental workload during visual search of symbolic displays. Appl Ergon 23:243–254. https://doi.org/10.1016/0003-6870(92)90152-l
Balconi M, Mazza G (2009) Brain oscillations and BIS/BAS (behavioral inhibition/activation system) effects on processing masked emotional cues: ERS/ERD and coherence measures of alpha band. Int J Psychophysiol 74:158–165. https://doi.org/10.1016/j.ijpsycho.2009.08.006
Balconi M, Mazza G (2010) Lateralisation effect in comprehension of emotional facial expression: a comparison between EEG alpha band power and behavioural inhibition (BIS) and activation (BAS) systems. Laterality 15:361–384. https://doi.org/10.1080/13576500902886056
Balconi M, Brambilla E, Falbo L (2009) BIS/BAS, cortical oscillations and coherence in response to emotional cues. Brain Res Bull 80:151–157. https://doi.org/10.1016/j.brainresbull.2009.07.001
Balconi M, Bortolotti A, Gonzaga L (2011) Emotional face recognition, EMG response, and medial prefrontal activity in empathic behaviour. J Neurosci Res 71:251–259. https://doi.org/10.1016/j.neures.2011.07.1833
Ball LJ, Eger N, Stevens R, Dodd J (2006) Applying the PEEP method in usability testing. Interfaces 67(Summer):15–19
Bialkova S, Grunert KG, van Trijp H (2020) From desktop to supermarket shelf: eye-tracking exploration on consumer attention and choice. Food Qual Prefer 81:103839. https://doi.org/10.1016/j.foodqual.2019.103839
Bruder C, Hasse C (2020) What the eyes reveal: investigating the detection of automation failures. Appl Ergon 82:102967. https://doi.org/10.1016/j.apergo.2019.102967
Buscher G, Cutrell E, Morris MR (2009) What do you see when you’re surfing? Using eye tracking to predict salient regions of web pages. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 21–30. https://doi.org/10.1145/1518701.1518705
Byrne MD, Anderson JR, Douglass S, Matessa M (1999) Eye tracking the visual search of click-down menus. In: Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pp 402–409. https://doi.org/10.1145/302979.303118
Capellini R, Tassistro F, Actis-Grosso R (2015) Quantitative metrics for user experience: a case study. In: International conference on product-focused software process improvement, pp 490–496. Springer, Cham. https://doi.org/10.1007/978-3-319-26844-6_36
Carver CS, White TL (1994) Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS scales. J Pers Soc Psychol 67:319. https://doi.org/10.1037/0022-3514.67.2.319
Chen L, Pu P (2010) Eye-tracking study of user behavior in recommender interfaces. In: International conference on user modeling, adaptation, and personalization, pp 375–380. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13470-8_35
Cloninger CR (1986) A unified biosocial theory of personality and its role in the development of anxiety states. Psychiatr Dev 3:167–226. https://doi.org/10.1016/0022-3956(92)90023-h
Corno F, Razzak F (2014) SAT based enforcement of domotic effects in smart environments. J Ambient Intell Humaniz Comput 5:565–579. https://doi.org/10.1007/s12652-013-0183-x
Cowen L, Ball LJ, Delin J (2002) An eye movement analysis of web page usability. In: People and computers XVI-memorable yet invisible, pp 317–335. Springer, London. https://doi.org/10.1007/978-1-4471-0105-5_19
Craig AR, Franklin JA, Andrews G (1984) A scale to measure locus of control of behaviour. Br J Med Psychol 57:173–180. https://doi.org/10.1111/j.2044-8341.1984.tb01597.x
Depue RA, Collins PF (1999) Neurobiology of the structure of personality: dopamine, facilitation of incentive motivation, and extraversion. Behav Brain Sci 22:491–517. https://doi.org/10.1017/s0140525x99002046
Doherty S, O’Brien S, Carl M (2010) Eye tracking as an MT evaluation technique. Mach Transl 24:1–13. https://doi.org/10.1007/s10590-010-9070-9
Duchowski AT (2003) Visual attention. Eye Track Methodol Theory Pract 1:3–15. https://doi.org/10.1007/978-1-4471-3750-4_1
Dumas JS, Redish JC (1999) A practical guide to usability testing (Revised, Subsequent ed.). Intellect Ltd.
Farma T, Cortivonis I (2000) Un questionario sul “locus of control”: Suo utilizzo nel contesto italiano (A questionnaire on the locus of control: its use in the italian context). Ricerca Psicoter 2:1–18
Fowles DC (1987) Application of a behavioral theory of motivation to the concepts of anxiety and impulsivity. J Res Pers 21:417–435. https://doi.org/10.1016/0092-6566(87)90030-4
Gable SL, Reis HT, Elliot AJ (2000) Behavioral activation and inhibition in everyday life. J Pers Soc Psychol 78:1135–1143. https://doi.org/10.1037/0022-3514.78.6.1135
Glöckner A, Herbold AK (2011) An eye-tracking study on information processing in risky decisions: evidence for compensatory strategies based on automatic processes. J Behav Decis Mak 24:71–98. https://doi.org/10.1002/bdm.684
Gram-Hanssen K, Darby SJ (2018) “Home is where the smart is”? Evaluating smart home research and approaches against the concept of home. Energy Res Soc Sci 37:94–101. https://doi.org/10.1016/j.erss.2017.09.037
Gray JA (1990) Brain systems that mediate both emotion and cognition. Cogn Emot 4:269–288. https://doi.org/10.1080/02699939008410799
Gray JA (1994) Personality dimensions and emotion systems. In: Ekman P, Davidson RJ (eds) The nature of emotion: fundamental questions. Oxford University Press, New York, pp 329–331
Gray JA (2003) A theory of the behavioural inhibition system. In: Gray JA (ed) The neuropsychology of anxiety. Oxford: Oxford Psychology Series. https://doi.org/10.1093/acprof:oso/9780198522713.003.0005
Green TM, Fisher B (2012) Impact of personality factors on interface interaction and the development of user profiles: next steps in the personal equation of interaction. Inf Vis 11:205–221. https://doi.org/10.1177/1473871612441542
Hekkert P (2006) Design aesthetics: principles of pleasure in product design. Psychol Sci 48:157–172. https://doi.org/10.1016/b978-008045089-6.50013-7
Henderson JM, Hollingworth A (1999) The role of fixation position in detecting scene changes across saccades. Psychol Sci 10:438–443. https://doi.org/10.1111/1467-9280.00183
Hooge ITC, Erkelens CJ (1996) Control of fixation duration in a simple search task. Percept Psychophys 58:969–976. https://doi.org/10.3758/bf03206825
Hooge ITC, Erkelens CJ (1998) Adjustment of fixation duration in visual search. Vis Res 38:1295-IN4. https://doi.org/10.1016/s0042-6989(97)00287-3
Hutchinson TE, White KP, Martin WN, Reichert KC, Frey LA (1989) Human-computer interaction using eye-gaze input. IEEE Trans Syst Man Cybern 19:1527–1534. https://doi.org/10.1109/21.44068
Hyönä J, Nurminen AM (2006) Do adult readers know how they read? Evidence from eye movement patterns and verbal reports. Br J Psychol 97:31–50. https://doi.org/10.1348/000712605x53678
Hyönä J, Laine M, Niemi J (1995) Effects of a word’s morphological complexity on readers’ eye fixation patterns. In: Studies in visual information processing, vol 6, pp 445–452. North-Holland. https://doi.org/10.1016/s0926-907x(05)80037-6
Jacobs AM (1986) Eye-movement control in visual search: how direct is visual span control? Perception and Psychophysics 391:47–58. https://doi.org/10.3758/bf03207583
Jacob RJ, Karn KS (2003) Eye tracking in human-computer interaction and usability research: ready to deliver the promises. In: The mind’s eye, pp 573–605. North-Holland. https://doi.org/10.1016/b978-044451020-4/50031-1
Kanemura A, Morales Y, Kawanabe M, Morioka H, Kallakuri N, Ikeda T (2013) Awaypoint-based framework in brain-controlled smart home environments: brain interfaces, domotics, and robotics integration. IEEE Int Conf Intell Robot Syst. https://doi.org/10.1109/IROS.2013.6696452
Lai ML, Tsai MJ, Yang FY, Hsu CY, Liu TC, Lee SWY, Tsai CC (2013) A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educ Res Rev 10:90–115. https://doi.org/10.1016/j.edurev.2013.10.001
Latour PL (1962) Visual threshold during eye movements. Vis Res 2:261–262. https://doi.org/10.1016/0042-6989(62)90031-7
Laugwitz B, Held T, Schrepp M (2008) Construction and evaluation of a user experience questionnaire. In: Symposium of the Austrian HCI and usability engineering group, pp 63–76. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89350-9_6
Leone L, Pierro A, Mannetti L (2002) Validità della versione italiana delle scale BIS/BAS di Carver e White (1994): generalizzabilità della struttura e relazioni con costrutti affini. Ita J Psychol 29:413–436
Levine JL (1984) Performance of an eyetracker for office use. Comput Biol Med 14:77–89. https://doi.org/10.1016/0010-4825(84)90022-2
Liu H, Heynderickx I (2011) Visual attention in objective image quality assessment: based on eye-tracking data. IEEE Trans Circuits Syst Video Technol 21:971–982. https://doi.org/10.1109/tcsvt.2011.2133770
Lopez-Aguila AA, Bustamante-Bello MR, Navarro-Tuch SA, Ramirez Mendoza RA (2020) Communication system development for emotional domotics interactive space. Int J Interact Des Manuf 14:27–736. https://doi.org/10.1007/s12008-019-00624-0
Lykins AD, Meana M, Kambe G (2006) Detection of differential viewing patterns to erotic and non-erotic stimuli using eye-tracking methodology. Arch Sex Behav 35:569–575. https://doi.org/10.1007/s10508-006-9065-z
Mekuria DN, Sernani P, Falcionelli N, Dragoni AF (2019) Smart home reasoning systems: a systematic literature review. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-019-01572-z
Merwin D (2002) Bridging the gap between research and practice. User Exp Winter. https://doi.org/10.1016/s0270-4021(03)12012-5
Mihajlović V, Grundlehner B, Vullers R, Penders J (2014) Wearable, wireless EEG solutions in daily life applications: what are we missing? IEEE J Biomed Health 19:6–21. https://doi.org/10.1109/jbhi.2014.2328317
Navarro-Tuch SA, Lopez-Aguilar AA, Bustamante-Bello MR, Molina A, Izquierdo-Reyes J, Curiel Ramirez LA (2019) Emotional domotics: a system and experimental model development for UX implementation. Int J Interact Des Manuf 13:1587–1601. https://doi.org/10.1007/s12008-019-00598-z
Ooms K, De Maeyer P, Fack V, Van Assche E, Witlox F (2012) Interpreting maps through the eyes of expert and novice users. Int J Geogr Inf Sci 26:1773–1788. https://doi.org/10.1109/sibgra.1999.805722
Quay HC (1988) The behavioral reward and inhibition system: the relevance of the neuropsychological theory of Jefrrey A. Gray. In: Bloomingdale LM, Sergeant J (eds) Attention deficit disorder — Criteria, cognition, intervention. Pergamon Press, Oxford, England
Rotter JB (1966) Generalized expectancies for internal versus external control of reinforcement. Psychol Monogr Gen Appl 80:1. https://doi.org/10.1037/h0092976
Salthouse TA, Ellis CL (1980) Determinants of eye-fixation duration. Am J Psychol. https://doi.org/10.2307/1422228
Shojaeizadeh M, Djamasbi S, Trapp AC (2016) Density of gaze points within a fixation and information processing behavior. In: International conference on universal access in human-computer interaction, pp 465–471. Springer, Cham. https://doi.org/10.1007/978-3-319-40250-5_44
Togami H (1984) Effects on visual search performance of individual differences in fixation time and number of fixations. Ergonomics 27:789–799. https://doi.org/10.1080/00140138408963552
Toker D, Conati C, Carenini G, Haraty M (2012) Towards adaptive information visualization: on the influence of user characteristics. In: International conference on user modelling, adaptation, and personalization, pp 274–285. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31454-4_23
Tsianos N, Lekkas Z, Germanakos P, Mourlas C, Samaras G (2009) An experimental assessment of the use of cognitive and affective factors in adaptive educational hypermedia. IEEE Trans Learn Technol 2(3):249–258. https://doi.org/10.1109/tlt.2009.29
Turner ML, Engle RW (1989) Is working memory capacity task dependent? J Mem Lang 28(2):127–154. https://doi.org/10.1016/0749-596x(89)90040-5
Volkman FC (1976) Saccadic suppression: a brief review. In: Monty RA, Senders JW (eds) Eye movements and psychological processes. Earlbaum, Hilsdale
Wee HJ, Lye SW, Pinheiro JP (2020) A fixation-click count signature as a visual monitoring enhancement feature for air traffic controllers. In: International conference on intelligent human systems integration, pp 221–227. Springer, Cham. https://doi.org/10.1007/978-3-030-39512-4_35
Wichansky AM (2000) Usability testing in 2000 and beyond. Ergonomics 43:998–1006. https://doi.org/10.1080/001401300409170
Funding
Open access funding provided by Università Cattolica del Sacro Cuore within the CRUI-CARE Agreement. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by FC, LA, and MB. The first draft of the manuscript was written by FC and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Ethical approval
The research was approved by the Ethical Committee institution of the Department of Psychology of Catholic University of the Sacred Heart, Italy. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Consent to participate
All participants voluntarily took part in the experiment and gave informed written consent for participating in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Cassioli, F., Angioletti, L. & Balconi, M. Tracking eye-gaze in smart home systems (SHS): first insights from eye-tracking and self-report measures. J Ambient Intell Human Comput 13, 2753–2762 (2022). https://doi.org/10.1007/s12652-021-03134-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-021-03134-8