Skip to main content
Download PDF

Everyday automation experience: a research agenda

Personal and Ubiquitous Computing volume 24, pages 725–734 (2020)Cite this article

Introduction

Automation is finding its way into many parts of everyday life. This is manifested by the increasing opportunities for end users to offload decisions to their home appliances, to hand over control to their cars, to step into automated trains, or to go for shopping at self-checkout stores. Supported by artificial intelligence, emerging automated services are getting closer towards human cognitive functions, as they are integrating analysis and decision based on the processing of large information sets [48]. In anticipation of the broad impact that these emerging technologies will have on our sphere of life, a reflective and systematic consideration is necessary that leverages their full potential in terms of user experience. While there is a long human factors research tradition on automation, scholars have for long concentrated on highly specialized professional work tasks for trained personnel, such as control center operators or pilots. With technological innovations and new use cases in domains such as transport, home automation, and retail, the goal of designing automation for a broader population has become more important. This transition of automation technology towards the everyday life thus has put the experience of “naïve users” into the center of attention.

This special issue contains selected scientific contributions from two workshops at the CHI 2019 and CHI 2020 conferences [17, 20]. The goal of these workshops was to investigate the requirements and design approaches for everyday automation experience. Participants within these workshops came from different domains of science and practice, which led to a first systematic discussion of the scope, already pre-existing knowledge and remaining issues of this newly involving field. The papers have a broad scope in terms of application fields, but they all have in common that they seek to deeply understand specific aspects of user experience and the design of everyday automation systems. The selection of papers within this special issue reveals the breadth of experience aspects that need to be considered when dealing with automation experience.

In the first paper, Thomas Lindgren, Vaike Fors, Sarah Pink, and Katalin Osz investigate the anticipatory aspect of automation experience, drawing on a 1.5-year ethnographic study where five families used research cars with evolving automated driving functions in their everyday lives. Their analysis focuses on people’s gradual adoption to automated cars. Lindgren et al. impressively uncover the involved processes that involve anticipating if users can trust automated driving functions, what the ADAS features will do, and what the longer term technological possibilities will be. They also show that this anticipatory UX occurs within specific socio-technical and environmental circumstances, which would not be easy to derive in experimental settings. This form of studying anticipation therefore offers us new insights into how people adopt automated driving in their everyday commuting routines.

Elçin Hancı, Joyca Lacroix, Peter A. M. Ruijten, Antal Haans, and Wijnand IJsselsteijn investigate the role of personal attitudes and motivation for the use of self-tracking technologies as a new form of automation experience that impacts our understanding of body and behavior. An analysis of the survey data together with data from motivation for self-tracking scale indicates that commitment to self-track positively correlates with autonomous motivation for tracking and negatively with controlled motivation. The research shows how self-tracking technology, as a novel automation experience, impacts users’ everyday behaviors. Furthermore, the paper discusses the relevance of interpersonal differences in attitudes and motivations towards continuous automated collection of personal data, and it highlights the methodological challenges for measurement.

Holger Klapperich, Alarith Uhde, and Marc Hassenzahl explore the trade-offs of the comfort achieved by everyday automation with the joyful experiences of interaction with a product. As an interesting result of their field research, they show that manual interactions have experiential benefits over more automated processes. Using the example of coffee making, Klapperich et al. present a method that considers potential experiential costs of everyday automation and strategies of how to design interaction with automation. They conclude that full automation may not necessarily be the ultimate goal for the design of everyday artifacts, as many activities can nurture enriched and more meaningful experiences. Furthermore, they propose to restart the discussion on levels of automation and thereby to bring in more strongly the aspect of the users’ experience instead of focusing on technological sophistication and unreflected efficiency.

Muneeb Imtiaz Ahmad, Ingo Keller, David Robb, and Katrin Lohan investigate cognitive load as a correlate of automation experience. They propose a framework to detect cognitive load by non-intrusively measuring physiological data from the eyes and the heart. Using purely such physiological data sources, Ahmad et al. show that different levels of cognitive load can be accurately predicted by means of their presented approach, and they outline how these insights could be applied to different application contexts in automation, robotics, and special vehicle operation. Attentive interfaces could build on this and further scientific accounts, in order to monitor the time-varying capabilities of users, in order to propose suitable manual interventions and handovers.

Carmelo Ardito, Giuseppe Desolda, Rosa Lanzilotti, Alessio Malizia, Maristella Matera, Paolo Buono, and Antonio Piccinno demonstrate and evaluate an approach how non-programmers can be supported to configure and orchestrate automated devices through the Internet of Things (IoT) for expressive and immersive end-user experiences. Ardito and colleagues experiment with different approaches that enable domain experts to identify and specify properties of IoT devices through user-defined semantics and thereby to facilitate them to automate the IoT devices’ behavior. In their study, they compare three different interaction paradigms that support the specification of user-defined semantics through a “transparent” use of ontologies. Furthermore, a set of conclusions is proposed with regard to how the proposed paradigms help domain experts express their semantics, which in turn facilitates the creation of interactive applications enabling smart interactive environments.

Alexander Georg Mirnig, Magdalena Gärtner, Elisabeth Füssl, Karin Ausserer, Alexander Meschtscherjakov, Vivien Wallner, Moritz Kubesch, and Manfred Tscheligi direct their attention to moments where automated operation comes to its limits. In their study on incident management in fully automated public transport, they explored participant’s behavior and willingness to assist in solving problems in a variety of scenarios where the bus suddenly stops for technical reasons or a hypothesized situation of harassment. The authors found that suggesting responsible behaviors is an existent barrier, when passengers are involved in the resolving of potentially dangerous situations in a public transport setting. The authors provide recommendation on how to address this challenge, by the multimodal design of incident-relevant instructions that are explicit, brief, timely, and distinguishable from regular on-trip information,

In the remainder of this editorial, we draw an agenda that shall function as a guiding frame for future everyday automation experience research endeavors. First, we highlight the special features of everyday automation experience that research should address, and then we look at the way how experience can be captured. We also highlight the benefits and potential limitations of investigating automation experience across various application domains. We then point towards promising design approaches that could help advance this field.

Addressing the special features of everyday automation experience

Human experience in general as well as user experience is a highly complex and entangled phenomenon. Thus, many scholars from different fields and research traditions have explored and conceptualized the various facets and the broad spectrum of experiences with technology in different contexts and applications domains. This richness and variety of research is, for example, represented in the subcommittee “User Experience and Usability” of the annual CHI conference. The experience of automated systems by non-professional users in everyday situations must be described and analyzed in a qualitatively different way than both conventional forms of human-machine interaction and professional automation use cases. In the following, we introduce into special features of automation experience (see Fig. 1), and for each of them, we highlight related open questions that need to be addressed to improve our understanding of automation experience.

Fig. 1
figure 1

Special features of everyday automation experience

Full size image

Uninitiated encounters

People increasingly encounter systems in their surroundings that they are not habituated or trained for. While first-time uninitiated usage is already a challenge with standard interfaces, this gets more delicate when a service or system works on its own, without explicit user commands. One of the involved challenges is that their behavior may not be as easily foreseeable as when operated by a human. For example, an automated vehicle approaching may be driving differently than what a pedestrian may have expected. Likewise, the automated handling of processes in the public, such as increasingly in airport check-ins or in supermarkets, is something that can strongly impact the overall experience. In order to improve intelligibility, automotive human factors research has investigated ways to realize awareness and intent communication, towards other road users or passengers have been investigated (see, e.g., Mirnig et al. from this special issue or [19]).

Implicit interaction

In everyday automation usage situations, continuous interaction is typically not as central as with manually operated systems. Relationships of users to automation in everyday contexts can be rather described as forms of “implicit interaction” [29, 56, 59] or peripheral interaction [4]. But while implicit and peripheral interaction is often thought to be in the direction that users provide inputs to a system, automated systems can often go further by taking initiative by addressing users with their requests and actions. How to make users understand “when it is their turn”, when they can override, and where they better follow the recommendation of a system are corresponding fundamental research questions. More in general, the question arises, what it means to experience ubiquitous automated systems if they are increasingly encountered indirectly or on the periphery of users’ attention.

Ambient embeddedness

Supported by new forms of implicit interaction, many automated systems do not anymore feature prominent displays, as they are merging into the users’ surroundings. In a sense, everyday automation is thus a strong driver for putting into practice the visions of “ubiquitous computing” [58, 73], the “disappearing computer” [66], as well as ambient intelligence [10]. The ambient nature of automated systems and their interwovenness in mundane, repetitive routines also supports the ordinariness of the involved user experience [11, 40]. Modern building management systems are a good example for underlying automation that frees users from complexity, but that can also be intransparent [1]. Also, here it is important to provide, when needed, cues on how to access more detailed information about what is going on and how building tenants can intervene in case they want to change the system behavior.

Decoupledness

Automation has been around us below the level of noticeability for a long time already, interweaving with critical infrastructure such as water and electricity supply, escalators, or conveyor belts. The now evolving forms of automation incorporate higher cognitive functions and through their behavior are more pervasively entering our sphere. Household energy management is a good example: many of us have become used to some form of heating regulation processes in the background. However, when demand-response systems remote control our boiler, the electric car, and other appliances in the household, we are being faced with a central instance that impacts more than just a subsystem, as it may have more overview over the own house than ourselves. However, even when appliances, transactions, and processes are completely automated, without direct user control, there is often still some remote monitoring of multiple devices. Examples for this are automated mobility services with control centers that intervene in case of an irregular situation. Questions related to this are how to provide a sense of potential remote coordination of automated services. Given the processing of data is done at other places, as is the case with today’s distributed and cloud-based smart assistants, research should also more systematically address privacy, security, and trust concerns of such forms of remote automation approaches.

Perceived system adaptation

With the rise of artificial intelligence, systems in the user’s surroundings are enabled to learn and consequently change their behavior. While the adaptation to user preferences can increase comfort, this may aggravate the above-mentioned challenge of intelligibility. Related issues that should be addressed are thresholds for perceivability and acceptance factors of system adaptation over longer time periods and the possibility of modifying adaptations when they are not effective.

Substitutive experiences

When an activity does not have to be performed any more, since it has become automated, people can do other things. For example, in a highly automated car, different types of replacement activities have been developed, reaching from relaxation and entertainment to work. In such safety-critical situations, replacement activities bear the risk of demanding too much attention, in case a manual handover is needed. While there are many promising approaches, more research needs to be undertaken to enable for engaging and comfortable replacement tasks while maintaining situation awareness. In other domains, such as the home and public areas, replacement activities have been addressed not so intensely. A more reflective consideration of the substitutive experiences evoked by replacements of activities by automation should be enabled also in these areas. For less safety-critical situations, requirements and design research could encompass ways to inspire people to turn to more challenging and engaging activities than passive entertainment or to recommend activities that fit to personal profile and interests.

Interventions

In many usage contexts, manual interventions have to be supported, in order to better adapt the system behavior to the respective context and preferences. Such interventions could make a system more useful, such as when the window blinds in the smart home are not behaving in the right way such that comfort is compromised. This way one more or less abruptly has to swap from a passive mode towards an expert both in using the system and the domain the system is operating in. In other, more safety-relevant cases, these interventions will call for quick and reliable human responses. The prime example for this is the control handover in automated driving, where the situation must be quickly grasped. Related challenges are to increase the user’s understanding of which interventions can be performed and which consequences they will entail.

End users as programmers

Apart from situation-dependent interventions, people can also get into a quite active role when it comes to the customization of automated systems. In fact, automation enables the potential for the creative planning and customization of system behaviors to people who have never programmed before. Many services, such as Amazon Alexa, offer means for end-user programming, such as the configuration of routines by If-This-Then-That methods. A central question in this regard is how to enable people with no or little programming skills to customize the behavior of a system. The possible solutions should allow users to understand whether what they specify actually corresponds to the desired behavior. Such solutions should be available for immediate and easy use, for example, by allowing users to specify their desired personalization rules through direct interactions with the objects involved in the automation. However, when non-programmers are supported, there also need to be mechanisms in place that safeguard the correct functioning of the system. Additionally, one needs to address the paradox challenge of both enabling informal end-user programming languages while still maintaining formality on the system’s side.

These special features of everyday automation each comprise their own research challenges. But it is also necessary to enable a more comprehensive understanding of their interrelationship and their dependence on contextual factors. When it comes to conceptualizing and capturing the specific forms and facets of everyday automation experience, challenges arise from these specific issues. Research on everyday automation experience has to address these specifics of everyday automation encounters characterized by interactions that are less direct and explicit as well as rather unobtrusively and peripherally perceived. For example, we need to develop and readily make available methods to capture user experience within the implicitness of everyday automation interactions, the peripheral encountering of automated systems, as well as the ordinariness of experiences with automation technologies. In the following section, approaches for investigating these issues are discussed.

Unpacking and capturing the facets of everyday automation contexts

A main challenge for future research on everyday automation experience is to explore and unpack the full spectrum of experiences with automation technology that might be specific for different domains and usage contexts. To unpack and appropriately capture the varying facets as well as the complexity of everyday automation experience, conceptual and methodological work is needed to provide an understanding of everyday automation experience across domains, especially as current predominant conceptions of user experience [25, 74] might fall short in framing and conceptualizing everyday automation experience and its special features.

Research on everyday automation experience needs to explicitly reflect on and articulate which kind of everyday automation experience is captured. Certain studies and research projects deploy different research designs, different approaches, different theoretical backgrounds, and different methods. This inherently leads to different facets, features, and forms of everyday automation experience that are captured. For this purpose, we refer to the framework introduced by Kuutti and Bannon [33] that distinguishes between an “interaction perspective” and a “practice perspective” in human-computer interaction and transfer it to the field of automation experience in order to articulate and structure different analytical perspectives on everyday automation experience.

Interaction perspective on automation experience

Building on Kuutti and Bannon’s [33] framework, automation experience research taking an “interaction perspective” centers around the very specific interactions between humans and automated technologies. This perspective focuses on the momentary and ahistorical experiences that arise from human-automation encounters and usually comes along with methods from psychology, for example, conducting controlled experiments in the lab and/or short-term studies focusing on users’ tasks. An interaction perspective might be particularly challenged by the “disappearing interaction” phenomenon related to ubiquitous automated systems but even so fruitful to address this phenomenon.

Practice perspective on automation experience

On the opposite, Kuutti and Bannon [33] propose a so-called “practice perspective,” which can be characterized by “decentering the human and the computer” [51]. Transferred to the exploration of everyday automation experience, research taking a practice perspective focuses on processes and long-term actions that are interwoven with the physical, social, and cultural surroundings. This perspective is often related to the deployment of qualitative methods, observational studies, long-term engagement with users in natural settings, and studies of activities surrounding the specific interaction of a human and an automated system. A “practice perspective” on automation experience might be specifically suitable to address the ordinary and mundane facets of everyday automation experience or to explore experiences related to the ambient embeddedness of automated systems.

We suggest relying on this framework of “interaction” and “practice perspective” to frame different analytical positions regarding everyday automation experience. It enables to distinguish and cluster the different angles of research on everyday automation experience as well as to articulate which analytical stance is taken to address, capture, and conceptualize certain features of everyday automation experience, thus rendering this differences visible. Certain research methods are particularly suitable for capturing specific forms of everyday automation experiences, and authors should reflect on how specific methods are applied and can be deployed to unpack and capture different features and nuances of everyday automation experience. This might provide a basis for further reflections and conceptual as well as methodological work on everyday automation experience.

Design concepts and development approaches for everyday automation experience

Researchers have started to work on generic design guidelines for automated systems. Examples include the concept of intervention user interfaces by Schmidt and Herrmann [57] and corresponding design principles derived from Shneiderman’s well-known golden rules [64]. Amershi et al. [3] proposed 18 design guidelines for human-AI interaction gathered from literature. While these principles and guidelines are generally applicable, we argue for an increased consideration of the characteristics of everyday automation in order to enhance the user experience of such appliances. In the following sections, we elaborate on promising research directions for improving interaction with, intelligibility of, and trust in ubiquitous automated systems.

Improved interventions

Despite advances in artificial intelligence, there will be situations, in which an automated system should sporadically enable interaction with their users. Such interactions include its configuration, potential interventions during the automated process, the confirmation of recommended actions, and rating and giving feedback on completed tasks. Fundamental for designing beneficial and user-accepted automated applications for everyday life, we consider a deeper understanding of the various application domains and users’ respective expectations and requirements. Application domains for everyday automation might widely differ regarding desired interaction opportunities. For example, public transport users might appreciate an automated service selecting and buying the most suitable ticket for their journey. However, this fully automated “action implementation” [50] might not be desirable in a restaurant or shopping context where users might probably experience a loss of control and feel overruled by system. Here, rather an “action selection” [50] in form of recommendations of a meal or product might be useful, however, the user might like to make the final confirmation based on additional context parameters not known to the system. Still, in both application domains, user interaction in the form of feedback after system usage is of value to further learn about the user preferences and adapt in future cases.

Improved intelligibility

The reasoning of (AI-driven) automated services and their underlying data processing are complex and hard to understand for non-expert users. This situation is worsened by systems involving a multitude of automated appliances. An example is the smart home domain, where users are able to combine virtually an unlimited number of smart gadgets which exchange data. An overall understanding of the mechanisms and reasoning of such smart systems seem a crucial factor for their adoption by non-expert users and for effective interactions with them. While the research area of “Explainable AI” aims at improving the transparency of AI-driven processes and at making them more comprehensible, recently introduced approaches mainly address expert users and are still too complex to support non-professional users.

In order to design for intelligible automated systems for everyday life, we argue that the users’ mental models of automated everyday objects need to be investigated. In related work, Tabassum et al. [67] explored users’ mental models of the data practices of their smart home devices. Today, many smart home gadgets and household robots come with mobile applications for their configuration. It seems worth exploring how such apps can provide meaningful supportive information during usage beyond simple status indicators. For example, augmented reality views might provide contextual information and guidance for automated systems with a physical presence appropriate for non-experts.

Maintained integrity

With regard to many of the above aspects, important questions on integrity and trust arise. This concerns trust into an external entity or body that may have too much insight or control into one’s own sphere of action but also trust in the automation system capabilities themselves. Researchers have pointed out the relevance of appropriate trust calibration to avert misuse, disuse, or abuse of an automated system [26, 49]. In case of an inappropriate level of trust, people may not accept or inadequately rely on autonomous systems in their everyday life [34]. Trust calibration is not an easy task for users, and therefore previous studies in professional have recommended that training on the task may be required for interacting with limited automation systems [41]. One central question is whether the factors affecting trust in professional contexts are similar as those in everyday automation. One of the important differences is that in everyday, contexts training is usually not considered to be important or practical. Furthermore, while users are often obliged to use certain technologies in professional settings, they can take advantage of opt-out opportunities in private life.

Opening the view across application domains

The above described trend towards everyday automation implies that it encompasses a broad spectrum of our experiences. Consequently, research on human-automation interaction will also have to continue to expand both regarding the breadth of investigated areas and the depth of knowledge about them [28]. In this regard, a key challenge is to find ways how a comprehensive and consistent experience for users can be enabled, without having to switch to, learn, and monitor many different paradigms. A prime example is the smart home, where the best way for managing the multitude of entertainment, security, energy, and housekeeping devices has not yet been found. For some aspects of automation experience, such as trust formation, an integrative perspective has been taken on by some scholars (see especially [34]).

However, there is still much potential with transferring principles drawn from professional usage scenarios (such as the handling of large aircraft automation systems) to less intensively trained usage (such as light aircrafts with amateur pilots). Furthermore, the question arises which design patterns proven successful in already intensively researched domains and can be applied to other application fields. Such efforts of holistic consideration will have to extend previous approaches of categorization into levels of automation across domains [15, 30, 60, 62, 70]. Some have started to implement these levels of automation in applications domains, such as in manufacturing [22], and more widely in the field of automated transport. Table 1 provides a brief and non-exhaustive overview of the status of everyday automation in some of the relevant application domains, and it lists respective topics and issues being focused in current investigations.

Table 1 Overview of application domains of everyday automation, the respective status of everyday automation, and the everyday automation aspects focused so far in research
Full size table

Towards a design space for everyday automation experience

Exploring and structuring the manifold facets of everyday automation seem necessary in order to provide a common basis for analyzing and discussing existing and upcoming appearances of everyday automation, for identifying research gaps as well as for facilitating cross-domain innovation and thus for inspiring future automated systems for everyday life. To this end, Shneiderman [63] recently presented a human-centered artificial intelligence framework that categorizes system types into a space along the levels of human and system control. He concludes that one should design both for high levels of human control and of system control. Bongard et al. [6] propose the definition of an “Automation Space,” a design space for everyday automation. Based on existing related categorizations [e.g. 49] and a structured review of scientific studies on automated systems in everyday life and its subsequent qualitative content analysis, the researchers identified eight core dimensions for categorizing ubiquitous automated systems: the presence of the system, the functionality, the context, the application domain, the type of automated task, the available interaction modalities, the required user expertise, and the level of automation. Figure 2 depicts these dimensions and their potential characteristics: e.g., the presence of the system might be either virtual or physical, and application domains include shopping, transportation, home, and user interactions that might be possible through voice, gesture, etc. For visualizing this design space, the authors propose a so-called morphological box: this matrix presentation features all dimensions and their characteristics in rows; thus, each automated system can be represented by a vertical path through these rows, and each novel path might inspire a future ubiquitous automated system. Figure 2 contains two example systems from literature such as an automatic vacuum cleaner robot.

Fig. 2
figure 2

“Automation Space,” a preliminary version of a design space for everyday automation with two examples of Bongard et al.

Full size image

Summary and conclusions

In this introduction of the theme issue, we have presented eight special features of everyday automation experience and propose related promising research directions. We highlight that users get into contact with automation experience technology in the form of uninitiated encounters and that they are exposed to implicit ambient interactions. Living with such systems requires people to trust in calm, often unnoticeable, distributed system operation and adaptation, but they also may necessitate abrupt manual interventions and knowledgeable customization activities. We argue that, while important theoretical concepts have been proposed for related aspects, a more comprehensive conceptual framing is needed to grasp the qualities of everyday automation experience in their entirety and their various manifestations.

Of central importance is how to collect and analyze everyday user experience data, and thus we also need to come up with suitable methodologies to unpack and capture them. We juxtaposed interaction and the practice perspectives, and we state that both are needed to achieve meaningful insights. Building up on this broader empirical foundation, we highlight three avenues for design, which shall improve intelligibility, interventions, and integrity of everyday automation experience. Thereby, a broad and informed view should be adopted, which takes account of approaches and achievements in other application domains. Ideally, the achieved design approaches should feed into further instances of the above introduced preliminary design space that could help shape everyday automation experience into something desirable for the future.

References

  1. Ahmadi-Karvigh S, Ghahramani A, Becerik-Gerber B, Soibelman L (2017) One size does not fit all: understanding user preferences for building automation systems. Energy Build 145:163–173

    Article  Google Scholar 

  2. Ali SMM, Augusto JC, Windridge D (2019) A survey of user-centred approaches for smart home transfer learning and new user home automation adaptation. Appl Artif Intell 33(8):747–774

    Article  Google Scholar 

  3. Amershi S, Weld D, Vorvoreanu M, Fourney A, Nushi B, Collisson P, Suh J, Iqbal S, Bennett PN, Inkpen K, Teevan J, Kikin-Gil R, Horvitz E (2019) Guidelines for human-AI interaction. In proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Computing Machinery, New York, NY, USA, Paper 3, 1–13

  4. Bakker S, Hoven E, Eggen B (2015) Peripheral interaction: characteristics and considerations. Pers Ubiquit Comput 19(1):239–254. https://doi.org/10.1007/s00779-014-0775-2

    Article  Google Scholar 

  5. Bodell O, Gulliksson, E (2016) Teleoperation of autonomous vehicle (Master’s thesis)

  6. Bongard J, Baldauf M, and Fröhlich P. Automation space: towards a design space for everyday automation (2020) In Proceedings of Automation Experience across Domains, Workshop in conjunction with 38th ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)

  7. Boy GA (2020) Human–systems integration: from virtual to tangible. CRC Press

  8. Braun M, Broy N., Pfleging B, Alt F (2017) A design space for conversational in-vehicle information systems. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (pp. 1-8)

  9. Burgess-Limerick R (2020) Human-systems integration for the safe implementation of automation. Mining, Metallurgy & Exploration, 1–8

  10. Chatzigiannakis I, De Ruyter B., Mavrommati I. (Eds.). (2019) Ambient Intelligence: 15th European Conference, AmI 2019, Rome, Italy, November 13–15, 2019, proceedings (Vol. 11912). Springer Nature

  11. Clemmensen T, Hertzum M, Abdelnour-Nocera A (2020) Ordinary user experiences at work: a study of greenhouse growers. ACM Trans Comput-Hum Interact 27(3):Article 16 (June 2020), 31 pages. https://doi.org/10.1145/3386089

    Article  Google Scholar 

  12. Dadhich S, Bodin U, Andersson U (2016) Key challenges in automation of earth-moving machines. Autom Constr 68:212–222

    Article  Google Scholar 

  13. De Ruyck O, Conradie P, De Marez L, Saldien J (2019) User needs in smart homes: changing needs according to life cycles and the impact on designing smart home solutions. In IFIP Conference on Human-Computer Interaction (pp. 536-551). Springer, Cham

  14. Edlund P, Holmner Härgestam A (2020) Customer relationship management and automated technologies: a qualitative study on chatbots’ capacity to create customer engagement

  15. Endsley MR (1999) Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics 42(3):462–492

    Article  Google Scholar 

  16. Fröhlich P, Sackl A, Trösterer S, Meschtscherjakov A, Diamond L, Tscheligi M (2018) Acceptance factors for future workplaces in highly automated trucks. In Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 129-136)

  17. Fröhlich P, Baldauf M, Meneweger T, Erickson I, Tscheligi M, Gable T, de Ruyter B, Paternò F (2019) Everyday automation experience: non-expert users encountering ubiquitous automated systems. In Extended Abstracts of the 2019 CHI conference on human factors in computing systems (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, Paper W25, 1–8. DOI:https://doi.org/10.1145/3290607.3299013

  18. Fröhlich P, Walker B, Donmez B, Riener A (2019) Special issue on recent advances in automotive user interfaces and interactive vehicular applications research: part 1 – novel interaction approaches. Int J Mobile Human Comput Interact 11(2):1–3

    Article  Google Scholar 

  19. Fröhlich P, Schatz R, Buchta M, Schrammel J, Suette S, Tscheligi M (2019) “What’s the Robo-Driver up to?” Requirements for Screen-based Awareness and Intent Communication in Autonomous Buses. i-com, 18(2):151–165

  20. Fröhlich P, Baldauf M, Palanque P, Roto V, Meneweger T, Tscheligi, M, Rot V, Meneweger T, Tscheligi M, Becerra ZM, Paterno F (2020). Automation experience across domains: designing for intelligibility, interventions, interplay and integrity. In Extended Abstracts of the 2020 CHI conference on human factors in computing systems (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. DOI:https://doi.org/10.1145/3334480.3375178

  21. Fröhlich P, Esterl T, Adam S, Kuch D, Yilmaz S, Katzeff C., Winzer, C (2020) Towards a social license to Automate in demand-side management: challenges, Perspectives and Regional Aspects. Proc. EnInnov 2020

  22. Frohm J, Lindström V, Stahre J, Winroth M (2008) Levels of automation in manufacturing. Ergonomia 30(3)

  23. Geeng C., Roesner F. (2019) Who’s in control? Interactions in multi-user smart homes. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-13)

  24. Ghazizadeh M, Lee JD, Boyle LN (2012) Extending the technology acceptance model to assess automation. Cogn Tech Work 14(1):39–49

    Article  Google Scholar 

  25. Hassenzahl, M (2010) Experience design: technology for all the right reasons. synthesis lectures on human-centered informatics 3, 1 (2017/04/13 2010), 1–95. DOI:https://doi.org/10.2200/S00261ED1V01Y201003HCI008

  26. Hoff KA, Bashir M (2015) Trust in automation: integrating empirical evidence on factors that influence trust. Hum Factors 57(3):407–434

    Article  Google Scholar 

  27. Hopkin VD, Wise JA (2018) Human factors in air traffic system automation. Automation and human performance: Theory and applications, 192

  28. Janssen CP, Donker SF, Brumby DP, Kun AL (2019) History and future of human-automation interaction. Int J Human-Comput Stud 131:99–107

    Article  Google Scholar 

  29. Ju W, Leifer L (2008) The design of implicit interactions: making interactive systems less obnoxious. DesignIssues 24(07 2008):72–84. https://doi.org/10.1162/desi.2008.24.3.72

    Article  Google Scholar 

  30. Kaber DB, Endsley MR (2004) The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task. Theor Issues Ergon Sci 5(2):113–153

    Article  Google Scholar 

  31. Kadir BA, Broberg O, da Conceição CS (2019) Current research and future perspectives on human factors and ergonomics in Industry 4.0. Comput Ind Eng 137:106004

    Article  Google Scholar 

  32. Kocaballi AB, Laranjo L, Coiera E (2019) Understanding and measuring user experience in conversational interfaces. Interact Comput 31(2):192–207. https://doi.org/10.1093/iwc/iwz015

    Article  Google Scholar 

  33. Kuutti K, Bannon L (2014) The turn to practice in HCI: towards a research agenda. In Proceedings of the SIGCHI conference on human factors in computing systems (CHI ‘14). ACM, New York, NY, USA, 3543–3552. DOI: https://doi.org/10.1145/2556288.2557111

  34. Lee JD, See KA (2004) Trust in automation: designing for appropriate reliance. Hum Factors 46(1):50–80

    MathSciNet  Article  Google Scholar 

  35. Leonardi N, Manca M, Paternò F, Santoro C (2019) Trigger-action programming for personalising humanoid robot behavior. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-13)

  36. Li X, Schroeter R, Rakotonirainy A, Kuo J, Lenné MG (2020) Effects of different non-driving-related-task display modes on drivers’ eye-movement patterns during take-over in an automated vehicle. Transport Res Part F: Traffic Psychol Behav 70:135–148

    Article  Google Scholar 

  37. Lindblom J, Alenljung B, Billing E. (2020) Evaluating the user experience of human–robot interaction. In Human-Robot Interaction (pp. 231-256). Springer, Cham

  38. Lo CH, Wang YW (2019) Constructing an evaluation model for user experience in an unmanned store. Sustainability 11(18):4965

    Article  Google Scholar 

  39. McCall R, Mcgee, F, Meschtscherjakov A, Louveton N, Engel T (2016) Towards a taxonomy of autonomous vehicle handover situations. In Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 193–200)

  40. Meneweger T, Wurhofer D, Fuchsberger V, Tscheligi M (2018) Factory workers’ ordinary user experiences: an overlooked perspective. Hum Technol 14(2):209–232. https://doi.org/10.17011/ht/urn.201808103817

    Article  Google Scholar 

  41. Merritt SM, Lee D, Unnerstall JL, Huber K (2015) Are well-calibrated users effective users? Associations between calibration of trust and performance on an automation-aided task. Hum Factors 57(1):34–47

    Article  Google Scholar 

  42. Millonig A, Fröhlich P (2018) Where autonomous buses might and might not bridge the gaps in the 4 A’s of public transport passenger needs: a review. In Proceedings of the 10th international conference on automotive user interfaces and interactive vehicular applications (AutomotiveUI ‘18). Association for Computing Machinery, New York, NY, USA, 291–297. DOI:https://doi.org/10.1145/3239060.3239079

  43. Millonig A, Haustein S (2020) Human factors of digitalized mobility forms and services. Eur Transp Res Rev 12(1)

  44. Mirnig AG, Meschtscherjakov A (2019) Trolled by the trolley problem: on what matters for ethical decision making in automated vehicles. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-10)

  45. Mirnig AG, Gärtner M, Wallner V, Trösterer S, Meschtscherjakov A, Tscheligi M (2019) Where does it go? A study on visual on-screen designs for exit management in an automated shuttle bus. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ‘19). Association for Computing Machinery, New York, NY, USA, 233–243. DOI:https://doi.org/10.1145/3342197.3344541

  46. Neumeier S, Wintersberger P, Frison AK, Becher A, Facchi C, Riener A (2019) Teleoperation: the holy grail to solve problems of automated driving? Sure, but latency matters. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 186–197)

  47. Nguyen T, Lim CP, Nguyen ND, Brown L, Nahavandi S (2019) A review of situation awareness assessment approaches in aviation environments. IEEE Syst J 13(3):3590–3603

    Article  Google Scholar 

  48. Palanque P (2018) Engineering automations: from a human factor perspective to design, implementation and validation challenges. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS ’18). Assoc Comput Mach New York, NY, USA 2:1–2. https://doi.org/10.1145/3220134.3223044

    Article  Google Scholar 

  49. Parasuraman R, Riley V (1997) Humans and automation: use, misuse, disuse, abuse. Hum Factors 39(2):230–253

    Article  Google Scholar 

  50. Parasuraman R, Sheridan TB, Wickens CD (2000) A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybern-Part A: Syst Humans 30(3):286–297

    Article  Google Scholar 

  51. Pierce J, Strengers Y, Sengers P, Bødker S (2013) Introduction to the special issue on practice-oriented approaches to sustainable HCI. ACM Trans Comput-Hum Interact 20(4):Article 20, 8 pages. https://doi.org/10.1145/2494260

    Article  Google Scholar 

  52. Reisinger MR, Schrammel J, Fröhlich P (2017) Visual languages for smart spaces: end-user programming between data-flow and form-filling. In 2017 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (pp. 165-169). IEEE

  53. Riener A, Gabbard J, Trivedi M (2019) Special issue of presence: virtual and augmented reality for autonomous driving and intelligent vehicles: guest editors’ introduction

  54. Rohr C, Ecola L, Zmud J, Dunkerley F, Black J, Baker E. (2016) Travel in Britain in 2035: future scenarios and their implications for technology innovation. In Innovate UK from https://www.rand.org/pubs/research_reports/RR1377.html

  55. SAE On-Road Automated Vehicle Standards Committee (2018) Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. Warrendale, SAE International

    Google Scholar 

  56. Schmidt A (2000) Implicit human computer interaction through context. Pers Technol 4(2):191–199

    Article  Google Scholar 

  57. Schmidt A, Herrmann T (2017) Intervention user interfaces: a new interaction paradigm for automated systems. Interactions 24(5):40–45

    Article  Google Scholar 

  58. Schmidt A, Kranz M, Holleis P (2005) Interacting with the ubiquitous computer: towards embedding interaction. In Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies (pp. 147-152)

  59. Serim B, Jacucci G (2019) Explicating “implicit interaction” an examination of the concept and challenges for research. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-16)

  60. Sheridan TB (1992) Telerobotics, automation and human supervisory control. The MIT press, Cambridge

    Google Scholar 

  61. Sheridan TB (2019) Individual differences in attributes of trust in automation: measurement and application to system design. Front Psychol 10:1117

    Article  Google Scholar 

  62. Sheridan TB, Verplank WL (1978) Human and computer control of undersea teleoperators. Tech rep MIT man-machine systems laboratory. MA, Cambridge

    Book  Google Scholar 

  63. Shneiderman B (2020) Human-centered artificial intelligence: reliable, safe & trustworthy. Int J Human–Comput Interact 36(6):495–504

    Article  Google Scholar 

  64. Shneiderman B, Plaisant C, Cohen M, Jacobs S, Elmqvist N (2016) Designing the user Interface: strategies for effective human-computer interaction, Sixth Edition, Pearson (May 2016)

  65. Srinivasan V, Koehler C, Jin H (2018) RuleSelector: selecting conditional action rules from user behavior patterns. Proc ACM Interact Mob Wearable Ubiquitous Technol 2(1):Article 35 34 pages

    Article  Google Scholar 

  66. Streitz N, Nixon P (2005) The disappearing computer. Communications-ACM 48(3):32–35

    Article  Google Scholar 

  67. Tabassum M, Kosinski T, Lipford HR (2019) “I don’t own the data”: end user perceptions of smart home device data practices and risks. In Fifteenth Symposium on Usable Privacy and Security ({SOUPS} 2019)

  68. Tigwell GW, Crabb M (2020) Household surface interactions: understanding user input preferences and perceived home experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-14)

  69. Trösterer S, Meschtscherjakov A, Mirnig AG, Lupp A, Gärtner M, McGee F, McCall R, Tscheligi M, Engel T. (2017) What we can learn from pilots for handovers and (De)skilling in semi-autonomous driving: an interview study. In Proceedings of the 9th international conference on automotive user interfaces and interactive vehicular applications (AutomotiveUI ‘17). Association for Computing Machinery, New York, NY, USA, 173–182. DOI:https://doi.org/10.1145/3122986.3123020

  70. Vagia M, Transeth AA, Fjerdingen SA (2016) A literature review on the levels of automation during the years. What are the different taxonomies that have been proposed? Appl Ergon 53:190–202

    Article  Google Scholar 

  71. van der Heiden RM, Janssen CP, Donker SF, Kenemans, JL (2020) The effect of cognitive load on auditory susceptibility during automated driving. Utrecht University

  72. Vitor R, Keller B, D'Angelo T, Azpurua H, Bianchi AG, Delabrida S (2019) Collaborative teleoperation evaluation for drones. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems (pp. 1-11)

  73. Weiser M (1991) The computer for the 21 st century. Sci Am 265(3):94–105

    Article  Google Scholar 

  74. Wright P, McCarthy J (2010) Experience-centered design: designers, users, and communities in dialogue. Synthesis Lectures Human-Centered Inform 3(1) (2017/04/13 2010):1–123. https://doi.org/10.2200/S00229ED1V01Y201003HCI009

    Article  Google Scholar 

  75. Zeng E, Roesner F (2019) Understanding and improving security and privacy in multi-user smart homes: a design exploration and in-home user study. In 28th {USENIX} Security Symposium ({USENIX} Security 19) (pp. 159-176)

Download references

Author information

Authors and Affiliations

  1. AIT Austrian Institute of Technology, Vienna, Austria

    Peter Fröhlich & Manfred Tscheligi

  2. Eastern Switzerland University of Applied Sciences, St. Gallen, Switzerland

    Matthias Baldauf

  3. University of Salzburg, Salzburg, Austria

    Thomas Meneweger & Manfred Tscheligi

  4. Philips Research, Eindhoven, Netherlands

    Boris de Ruyter

  5. Radboud University Nijmegen, Nijmegen, Netherlands

    Boris de Ruyter

  6. C.N.R.-ISTI, Pisa, Italy

    Fabio Paternó

Authors
  1. Peter Fröhlich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthias Baldauf
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Meneweger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Manfred Tscheligi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Boris de Ruyter
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fabio Paternó
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Fröhlich.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fröhlich, P., Baldauf, M., Meneweger, T. et al. Everyday automation experience: a research agenda. Pers Ubiquit Comput 24, 725–734 (2020). https://doi.org/10.1007/s00779-020-01450-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-020-01450-y