Skip to main content

Human-Computer Interaction in Physical Retail Environments and the Impact on Customer Experience: Systematic Literature Review and Research Agenda

Part of the Lecture Notes in Computer Science book series (LNISA,volume 11588)


Shopping in the traditional retail environment is increasingly being influenced by technologies enabling human-computer interaction. Scientists, as well as retailers, are interested in examining the effects of interactive in-store technologies on customer behavior. An abundance of researchers have examined customer acceptance of interactive technologies, but little is known about its effects on the customer’s experience and its consequences, such as satisfaction, loyalty, and purchase intention. Following the guidelines for a systematic literature review, this article gives an overview of empirical studies conducted on interactive technologies in traditional brick-and-mortar retail stores. Based on that review, this paper provides a research agenda for future work on interactive in-store technologies and their impact on customer experience and suggests possible research methods for empirical studies in the field of human-computer interaction.


  • Customer experience
  • Interactive technology
  • Digital retail
  • Research methods

1 Introduction

In recent years, the retail industry has experienced increased pressure from competitors, not only in the traditional retail environment but also from online market players. As the online and offline retail worlds converge, traditional brick-and-mortar stores “will change into connected stores, serving as places for inspiration, experiences, showcases, and service centers” (p. 17) as Jongen [1] claims. According to a recent study by Capgemini Consulting [2], customers expect the same customer experience from traditional retail stores as from online shops. KPMG [3] state in their 2018 retail industry trend report that “[t]he customer experience is more important than ever as retailers are striving to differentiate themselves in a challenging and crowded market” (p. 4). The topic of technologies in the retail industry has been even more discussed in research and practice since Amazon’s announcement of plans to open more than 3,000 self-service stores by 2021 [4]. In science, Grewal et al. [5] reveal the main topics of the future of retail, and Priporas et al. [6] conceptualize a future agenda for digital retailing. All parties agree on the central aspect: Customers demand a superior and engaging experience in an interactive retail environment [6,7,8,9,10,11,12]. According to Roy et al. [11], the digital retail environment can be defined as “[a]n interactive and connected retail system which supports the seamless management of different customer touchpoints to personalize the customer experience across different touchpoints and optimize performance over these touchpoints” (p. 259).

Retailers and brands increasingly use technology to provide an engaging shopping experience in the physical retail environment [13]. For example, self-service checkout terminals can be found in almost every supermarket branch. This allows users to quickly scan and pay for products on their own [7] or easily retrieve detailed product information by scanning the product’s barcode [9]. Willems et al. [12] address three major benefits of technology in retail stores: beyond (1) time and cost savings, customers gain (2) utilitarian as well as (3) hedonic shopping value. First, using self-service technologies helps customers to save time because they can proceed without spending time waiting for a salesperson or queuing at the service counter [9, 11]. Moreover, it is possible to monitor and control expenditure in the store [8]. Second, customers derive utilitarian benefits from a shopping trip when accomplishing an intended task [10], such as gaining more knowledge about the product and comparing different variations [12]. Third, interactive in-store retail technology in combination with fun and engaging tools enhance the customer’s hedonic shopping value. For example, smart mirrors with augmented reality [10] or monitors with gamification mechanisms [14] provide such appealing experiences.

An abundance of researchers has examined the user acceptance of new technologies in a digital retail environment by using well-known theories, such as the theory of reasoned action (TRA) [15] or the technology acceptance model (TAM) [16, 17]. For example, Roy et al. [11] investigated the moderating effect of technology readiness on the customer’s attitude and behavioral intentions when using smart technologies, such as smart checkouts, personal shopping assistance, and augmented reality. Other researchers examined the application, functionality, and adoption of RFID (Radio Frequency Identification) equipped retail stores in a complex field study [18]. Willems et al. [12] extensively reviewed and summarized 54 digital retail technologies from various manufacturers along the customer’s path of purchase.

Following Parasuraman’s [19] pyramid model of marketing services, retailers need to consider that technology mediates nearly any communication between the three dimensions involved in the physical retail environment: the company, its employees, and the customers. Human-computer interaction is intensively discussed in the literature of retailing, consumer services, business, and information systems. However, empirical findings are limited considering the impact of interactive technologies in physical retail stores on the customer’s experience and its consequences, such as customer satisfaction [7, 10, 14, 20, 21], loyalty [22], intention to reuse [9], (re)patronage [10, 14, 23,24,25], word-of-mouth [26], and purchase intention [27]. In consideration of these criteria, this paper presents a systematic literature review based on 15 identified empirical studies.

The structure of this paper is organized as follows: Sect. 2 presents the methodology of this work, which is based on the guidelines for systematic literature reviews of vom Brocke et al. [28]. In the following Sect. 3, we conduct the review by defining the review scope, describing the literature search, and how the studies are analyzed and synthesized. Afterward, Sect. 4 presents the results of the systematic literature review, categorized by technology usage, research methods, and the applied retail context. The same classification is adopted to discuss the future agenda and further research possibilities (Sect. 5). Finally, the conclusion can be found in Sect. 6.

2 Methodology

This work follows the guidelines for conducting systematic literature reviews in the field of information systems (IS) proposed by vom Brocke et al. [28]. The authors claim that a literature analysis must be “comprehensibly described” as researchers should be able to use the same data for their own research purposes. By analyzing previous work of IS researchers, vom Brocke et al. [28] found that most of the reviewed publications leave gaps in the process documentation. Therefore, the guidelines on systematic literature reviews have been developed and are now used in this present work. The authors define five phases to the literature review: (1) scoping the review, (2) conceptualizing the topic, (3) searching appropriate literature, (4) analyzing and synthesizing identified literature, and (5) stating the review’s contributions to a research agenda.

In the first phase, researchers should define the scope of their literature review. The use of the six characteristics of Cooper’s [29] taxonomy, which highlight some of the central aspects of the review, is recommended. In the second phase, vom Brocke et al. [28] state that researchers must provide key definitions relevant to the topic. Therefore, we describe the concepts of customer experience and interactive technologies in physical retail stores. The third phase reveals the exact process of the peer-reviewed literature search and provides the keywords used in digital libraries. Moreover, this phase outlines the including and excluding criteria as well as the total number of recognized publications. After we have identified the relevant literature, we applied phase four and analyzed the studies in relevance to the technologies, the research methods and the retail context. The fifth phase of vom Brocke et al.’s [28] guidelines comprises the research agenda and presents several approaches and questions for future research.

This present work contains all five phases of the systematic literature review of vom Brocke et al. [28] and accumulates the current state of empirical research on interactive technologies in traditional brick-and-mortar stores.

3 Systematic Literature Review

The following section will outline in detail the focus of our systematic literature review by defining the review scope and the key terms, describing the literature search process and how we conducted the analysis.

3.1 Definition of the Review Scope

We defined the taxonomy of our search query with reference to Cooper [29], who proposed to classify the review to the following six major characteristics which are shown in Table 1: goal, focus, organization, perspective, audience, and coverage. (1) Our goal is to integrate and summarize empirical studies (journals and conference proceedings) on the customer’s experience arising from the use of interactive retail technologies. (2) Therefore, the focus is on the research methodology (qualitative, quantitative, or mixed) and the different physical retail environments (e.g., supermarket, fashion store) in which the studies have been applied. (3) The organization of the review follows a conceptual process in which the studies are initially organized according to the technologies’ application scenario (e.g., self-service, shopping assistance, store atmospherics). Afterward, we categorized the reviewed articles by the research methods used (e.g., laboratory or field experiments, surveys). (4) A neutral perspective is chosen to summarize and synthesize relevant empirical studies, but no critical assessment of the research is conducted. (5) The relevant audience for this paper primarily consists of researchers in the field of IS and human-computer interaction. (6) Finally, we cover an exhaustive literature review on empirical studies examining the use of interactive technologies in physical retail environments in terms of customer experience and its consequences, such as purchase intention and loyalty.

Table 1. Scope of Review (highlighted in Grey) using the Taxonomy proposed by Cooper [29]

3.2 Conceptualization of Topic

Following the guidelines of vom Brocke et al. [28], the key terms and the theoretical background of customer experience and interactive technologies will be provided.

Customer Experience.

In the 1960s, researchers started to investigate the customer’s decision-making process and the phenomena influencing customerbehaviorin physical retail stores [30]. Holbrook and Hirschman [31] were among the first to recognize the consumer’s “pursuit of fantasies, feelings, and fun” (p. 132), discussing emotionally rather than rationally driven customer choices. Ever since, however, marketing literature appears to develop from understanding the customer’s behavior to creating a customer-centric experience [30]. The work of Verhoef et al. [32] defines customer experience as a multidimensional construct that is “holistic in nature and involves the customer’s cognitive, affective, emotional, social and physical responses to the retailer” (p. 32). Furthermore, the authors argue that “[t]his experience is created not only by those elements which the retailer can control (e.g., service interface, retail atmosphere, assortment, price) but also by elements that are outside of the retailer’s control (e.g., influence of others, purpose of shopping)” (p. 32). Several researchers have investigated that the customer experience influences satisfaction and loyalty, both being crucial preconditions for a higher rate of shopping visits, higher spending and the retailer’s growth [8, 30]. Recently, the consideration of technology in multi- and omnichannel environments has played a major role in enabling retailers to provide a superior customer retail experience [8, 30].

Interactive Technologies.

The use of interactive technologies in the retail sector describes the communication between the customer and the retailer (the company and its employees) based on internet-enabled devices. The goal is to respond to customer needs by providing the requested service without any human interference [33]. Researchers intensively investigate self-service technologies such as order terminals and checkout systems, which allow users to experience service support or accomplish tasks without the involvement of service employees [34]. Interactive technologies offer a more engaging and superior shopping experience for customers in retail stores and beyond, by connecting channels with each other, such as interactive fitting rooms that would link to social networks [35]. Poncin et al. [10] claim that new digital technologies are key to enhancing the appearance of physical retail stores and lead to higher visitor rates as well as increased sales. Therefore, the use of interactive technologies results in an enhanced customer retail experience [10], strengthens customer engagement and generates a competitive advantage [36].

3.3 Literature Search

The search query covered interactive technologies (using the keywords “technology” or “ technologies” or “smart” or “interactive” or “digital”) in the context of customer experience (using the keywords “experience” and (“shopping” or “retail” or “customer” or “customers” or “consumer” or “consumers”). We included all types of English publications (e.g., research articles, proceedings papers, etc.) in the field of information and communication technology and business, the latter comprising the subcategory of marketing [12]. Based on the approach of Willems et al. [12] and due to the high-speed development of technologies, the ideal timeframe to review innovative in-store technologies covers 8.5 years, hence this present review contains scientific publications from July 2010 until December 2018. The search was conducted in the libraries of ScienceDirect (total number of results: 403; the number of results after the first review: 50; the number of results after the second review: 20), Web of Science (199; 55; 21) and Springer’s disciplines of business and management (600;20;0) and computer science (263;14;0). In order to identify retail-related publications, we first reviewed the title and keywords and second, excluded duplicates and analyzed the abstracts of 133 remaining publications. Subsequently, we used these publications to conduct a forward and backward search according to vom Brocke et al. [28]. Finally, we identified 15 studies that met our criteria, having investigated the use of interactive technologies and their influence on the customer’s experience in brick-and-mortar retail stores.

3.4 Literature Analysis and Synthesis

According to the guidelines of systematic literature reviews, we organize and systematically analyze the papers collected in the literature review on the technologies’ application scenario, methodology and research design, as well as on the retail context.

Application Scenario.

We classify the identified empirical studies by the application scenario. First, we summarize self-service checkout technologies. Second, we group shopping assistance technologies, such as smart fitting rooms and self-scanning systems. The third group comprises all technologies influencing the store’s atmosphere, such as smart mirrors and interactive window displays.

Methodology and Research Designs.

The consideration of research methodologies (qualitative, quantitative, and mixed) and designs (e.g., experiments, focus groups, and surveys), which have been used to conduct research on the interaction with technologies in physical retail stores.

Retail Context.

Finally, the literature review will show in which retail contexts the empirical investigations were conducted, such as supermarkets, fashion, or do-it-yourself (DIY) retail stores.

4 Results in the HCI Context

In this section, we provide the results of our systematic literature review. We follow the procedure proposed by vom Brocke et al. [28]. First, we describe the studies used in terms of interactive technologies and their influence on the customer experience in brick-and-mortar retail stores. Second, we report the research methodology used and third, the retail context in which the 15 empirical studies were conducted.

4.1 Interactive Technologies and Their Impact on Customer Experience

Self-service Checkout Technologies.

The majority of the identified papers focus on the evaluation of self-service checkout technologies. Retailers increasingly implement checkout terminals in their physical retail environments, which allow customers to quickly scan and pay for products on their own instead of queuing at the cashier desk [7, 9, 20]. In addition to the speed advantage, researchers involve further attributes to determine the perceived service quality of self-checkouts [23, 37]. These attributes include the ease of use, reliability, fun/entertainment, and perceived control. Another approach to measuring the service quality of self-checkout systems is to use standard scales, such as the SSTQUAL [22], which consists of seven dimensions (functionality, enjoyment, security, assurance, design, convenience, and customization). Reliability (individual and as a part of functionality) has been identified as a main [22] and even the most important [23, 37] driver of the customer’s perceived service quality of self-service checkout technologies. Consequently, service quality positively influences customer loyalty [22], purchase intention [37], intention to reuse [9, 38], and repatronage intention [23], directly as well as through the mediating effect of customer satisfaction. On the other hand, if customers are forced to use self-service checkout systems and gain the feeling of unfairness and dissatisfaction (due to no/little technology readiness or the need for human interaction), customers will be more likely to spread negative word of mouth, reduce future spending, and develop intentions to become the competitor’s customer [26]. Therefore, researchers suggest that employees should personally help to introduce new technologies to customers in order to create a positive customer experience, which will in turn, lead to higher satisfaction with the self-service checkout technology as well as with the store [7]. Satisfaction is considered one of the main drivers of the retailer’s revenue performance, store appeal, patronage, positive word-of-mouth, and customer loyalty [7, 12, 38, 39].

Shopping Assistance.

Self-scanning devices can be used with barcodes of products in order to receive enhanced product information, ratings from other customers, and recommendations for complementary products [7]. Moreover, the additional decision support features allow customers to manage and control their spending in the store [7], gain detailed product information, compare prices, check recommendations and ratings by other consumers [9, 13]. The advantages of self-service technologies enhance customer experience and subsequently overall satisfaction with the store [7]. However, retailers need to consider that the interpersonal service quality and the service quality of the self-scanning technology moderates the relationship between the customer and retail patronage [25]. This can be critical when the need for human interaction and technology anxiety are high.

Smart fitting rooms can support product choices by providing additional product information and recommendations [27]. The technology in the background of smart fitting rooms is the combination of RFID chips attached to the price tags of fashion articles and sensors installed in fitting rooms [18, 27]. The sensors are able to detect the products and display detailed information on the monitor, such as the brand’s name and logo, as well as the price of the product [27]. If retailers provide details of their security policy, customers appreciate the possibility to enter personal data at the monitor for an enhanced and more personalized experience. This study shows, that especially in the case of low-quality brands, it is advisable to provide data security disclosure. Moreover, higher interaction and fewer security concerns lead to higher satisfaction and result in stronger purchase intentions [27].

Multimedia kiosks in physical retail stores deliver entertaining information and allow customers to perform transactions [21]. Research shows that customers gain the feeling of perceived control and convenience by performing tasks on their own, which consequently results in satisfaction and the intention to reuse the technology [21].

Store Atmospherics.

Smart mirrors are considered as store atmospherics and are based on the technology of augmented reality [40]. A human-sized touch-screen monitor pictures the person standing in front of it. Customers are able to virtually try on digital clothing items by selecting and moving them on the touch-screen [41]. The holistic perception of a play store atmosphere can be supported by game terminals and magic mirrors where children can engage in an interactive playing scenario [10]. The positive evaluation of store atmospherics influences the customer’s perceived shopping value and positive emotions, which result in higher customer satisfaction and greater likelihood of future patronage and recommendation [10]. The gamification aspect also works for adults by stimulating positive feelings and enhancing the experience at the moment of play [14]. The results of the study showed, that the combination of gamification and augmented reality affects the likelihood of future patronage but cannot enhance the quality of the overall experience.

Besides in-store atmospherics, researchers examined the effect of exterior atmospherics by installing interactive window displays [42]. The main purpose of installing interactive screens in the shop window is to attract the customer’s attention and to motivate them to enter the store. Once the customer recognizes the display, it provides hedonic impressions (e.g., pictures and videos of white beaches) and presents utilitarian information (e.g., text descriptions of a travel destination), as well as insights into what the customer can expect in the store [42]. Obviously, the retail context is significant, as interactive window displays have a strong negative effect on perceived pleasure in terms of tourist travel bureaus, which sell local goods [24]. However, for tourist travel shop contexts only, the study found that customers are seeking for a high-contact and task-oriented shopping experience.

4.2 Methodology and Research Designs

We reviewed 15 empirical studies about interactive technologies and their influence on the customer experience in brick-and-mortar retail stores. As shown in Table 2, twelve publications used a quantitative questionnaire (quan), two used a mixed methods approach (MM) and one author conducted a focus group interview, which is considered a qualitative method (qual). Most of the studies used online or offline surveys, laboratory or field experiments as research design.

Table 2. Results of the literature review conducted in accordance with the guidelines of vom Brocke et al. [28]

4.3 Retail Context

The results of the literature review indicate that nine of 15 studies conducted their research in food retail environments. In the case of non-food retail scenarios, researchers used laboratory environments or surveys. No field experiment has yet been conducted in a non-food retail store, such as a fashion, decoration, or sports shop.

5 Discussion and Research Agenda

The research agenda of this systematic literature review provides an overview of empirical studies which have examined technologies used for human-computer interaction in brick-and-mortar retail stores. In the following section, we demonstrate research gaps and promote various possibilities for further research.

5.1 Interactive Technologies

From the practitioners’ and researchers’ perspectives, we see a high level of interest in the examination of customers behavior with interactive technologies in physical retail stores, as these usually require high expenditure and investment. Researchers should support retailers by examining how or even whether customer experience can be enhanced in specific retail environments through the installation of human-computer interaction enabling technologies. For example, interactive digital shopping walls allow customers to order products on a touch-screen monitor, which will later be delivered to the customer’s home or prepared for in-store pick up [12]. Modern checkout-processes include mobile payment as well as biometric authentication payment technology, such as face recognition or fingerprint [12]. Amazon presented a cashierless self-service checkout system called “Amazon Go” [5]. In Amazon’s case, store visitors identify themselves at the entrance by scanning a code in their smartphone app. Customers can pick up all the items they want and leave the store without any interference, either from a salesperson or a checkout terminal. According to Grewal et al. [5], this high-tech self-service technology, supported by a combined system of computer vision, sensor fusion, and deep learning technology, is what customers expect from shopping in the future. Artificial intelligence (AI), such as chatbots and AI-powered conversational interfaces, offers a whole new research field in the field of human-computer interaction and customer behavior. For example “Roberta” is a service-robot, working with chatbot technology, implemented to welcome customers, collect their inquiries as well as arrange appointments with a human service manager in the retail store [43].

Further investigation of the impact of interactive in-store technologies is recommended for several reasons. First, retailers are aware of the need for action to enable a superior and engaging customer experience in traditional brick-and-mortar stores. Therefore, scientifically proven results on the effects of technology on customer behavior are highly relevant for retailers. Second, the field of human-computer interaction in retail stores offers various research possibilities due to the number of different interactive technologies and retail sectors. Third, influence factors such as age, gender, countries, technology readiness, education, and motivation to shop, can determine in which retail environment the application of interactive technologies is recommended or discouraged.

5.2 Methodology and Research Designs

The great majority of the empirical studies in this review used quantitative questionnaires to measure the customer’s post-purchase experience evoked by using interactive technology in physical retail stores. Most of these studies were conducted in a laboratory or field experiment, as well as in online or offline survey scenarios. Quantitative questionnaires deliver numerical results, which researchers can easily use to calculate the influence of one variable on another, generalize and compare results, reuse the data for other research purposes, etc. [44]. However, on the other hand, quantitative results neither provide a deep insight into the topic nor reveal strong evidence for causal relationships [44]. Moreover, the feedback is based on the customer’s self-assessment of the human-computer interaction. Due to possible judgment errors, the self-evaluation might lead to deviations between the actual behavior and the answers given in the questionnaire [45]. Therefore, we suggest considering alternative measurement techniques from the field of Neurobiological Information Systems (NeuroIS), such as the ones below. These methods are usually applied in experimental settings [46] and allow researchers to identify responses to stimuli based on the nervous system [47].

  • Eyetracking records the customer’s eye movements and where they focus on. The results are usually aggregated and displayed on a heat map, which reveals which points were of high or low interest. Additionally, eyetracking can be combined with observation data or the “think aloud” technique in which participants comment on what they are doing, during or after the experiment. [48]

  • Electrodermal (or galvanic skin response) measurement is a technique from the field of psychophysiology. It measures the sweat glands in the palms of the hands and soles of the feet. It is possible to examine various activities with the electrodermal measurement method: arousal, stress, coping, information processing or decision-making. [49]

  • Electroencephalography (EEG) measures the electrical brain activation from the surface of the scalp. [50] A similar technique is called “functional magnetic resonance imaging” which measures brain activity by recognizing which areas are more active than others according to bloodstreams [47]. The measurement technique of EEG is often applied to investigate the performance of cognitive tasks with human-computer interaction system of an individual and serves as knowledge basis for the development of such systems [51].

According to Riedl and Léger [46] “[i]t is very common to use psychometric measures (i.e. surveys) in addition to neurophysiological measures in NeuroIS studies” (p. 108). Although one could assume that neurobiological measurements are more objective or reliable, they also have their weaknesses. This is, for example, highlighted by Riedl et al. [52] who state, that we should not assume that “neurobiological measurements are (necessarily) stable across repeated measurement and hence (perfectly) reliable [as] they are not” (p. xi). Various circumstances might influence the body’s measured values, such as situational factors (e.g., the participant is nervous because of the laboratory environment) or the experience and skill of the respective experimenter [52]. Other constructs which need to be considered with neurobiological measurements in empirical research of information systems are validity, sensitivity, diagnosticity, objectivity, and intrusiveness as proposed and discussed in detail by Riedl et al. [52]. Besides the neurobiological approach, researchers use cardiovascular or biochemical methods to examine phenomena in empirical studies. Cardiovascular measures assess the participant’s heart rate or blood pressure, which can be indicative of a stressful situation, whereas biochemical methods ascertain the hormone levels (e.g., cortisol) in saliva, blood or urine [53].

Technology development in the retail environment also leads to new measurement techniques for customer behavior in retail stores and evaluates implications to enhance the customer’s experience. Various customer tracking possibilities are enabled through the customer’s smartphone and near field communication (NFC) technology, global positioning systems (GPS) or wireless internet connection (Wi-Fi) in the local area network (LAN) [33]. These technologies are already commonly used in many retail stores and provide information on the customer’s behavior. Attention estimation measure methods use camera or sensor (e.g., infrared) technology to assess if customers are in the state of “ignoring”, “watching” or “ready to interact” with a stimulus [54]. Video observation is increasingly used to detect in-store shopping behaviors, such as walking (e.g., direction, speed, paths, stopping, queuing, etc.), the product choice (browsing, touching, holding, buying, etc.), socializing (people on the phone, with personnel or other customers, etc.), assistance behaviors (smartphone consultation, using hand-held scanners, using a shopping list, etc.), or other behaviors (redeeming coupons, eating, parking carts and baskets, etc.) [55]. Emotion recognition by the customer’s facial expression is based on video technology, such as Microsoft’s Kinect v2 [56] or Noldus’ FaceReader [57]. These devices are able to recognize the customer’s head pose and emotions, such as happiness, anger, surprise, or a neutral facial expression [57]. The data are used to inform managerial decisions, for example towards the products, layout, and atmospherics in the physical retail store [56, 57].

These gaps should be addressed in future works by combining both, technology-based and traditional research methods, to gain a holistic view of the effects of human-computer interaction in physical retail stores. “Roberta” the before mentioned service-robot does not collect the customer’s inquiries, but analyses the environment with integrated cameras and sensors [43]. Future research could investigate the customer’s behavior by analyzing the data logs of chatbot “Roberta” and combining them with a quantitative questionnaire after the customer’s experience in the store.

5.3 Retail Context

So far researchers have not yet examined interactive retail technologies in a non-food store field experiment. It can be assumed that most studies used self-checkout systems in a food store environment as this application scenario is already well established in practice. Future research questions should focus on the in-store human-computer interaction in different retail contexts, especially those that face increasing pressure coming from online commerce, such as fashion, active wear, sports equipment, cosmetics, and accessories.

6 Conclusion

The field of human-computer interaction has seen that investigating interactive technologies in a physical retail environment results in a better understanding of the customer’s experience and its consequences. These consequences refer to constructs such as customer satisfaction, loyalty, intention to reuse, patronage, word-of-mouth, and purchase intention. Our systematic literature review reveals first, that researchers should focus on the examination of interactive in-store technology to support retailers in their decisions on the appropriate choices in their physical retail stores. Second, the identified studies are mainly concerned with self-service checkout services examined by the customer’s self-assessment through quantitative questionnaires. We propose to extend the scientific literature on the customer’s in-store experience by using neurobiological, cardiovascular or biochemical measure methods complementary to traditional research methods in laboratory experiments, such as surveys or interviews. Moreover, using technologies (e.g., video cameras) to observe the customer’s behavior with in-store technologies (e.g., smart mirrors) could also offer a deeper insight into the customer’s perception of the experience in physical retail environments. Finally, our work proposes to cover many different retail scenarios, beyond food retail, such as fashion and cosmetics. In summary, researchers are encouraged to conduct studies on retail technologies by applying it to different technologies, research methods, and retail industries.

Change history

  • 10 July 2019

    The chapters were mistakenly published as regular chapters instead of open access. This has now been corrected.


  1. Jongen, W.: The End of Online Shopping: The Future of New Retail in an Always Connected World. World Scientific Books (2018)

    Google Scholar 

  2. Capgemini Consulting: Making the Digital Connection: Why Physical Retail Stores Need a Reboot (2017).

  3. KPMG International Cooperative: Global retail trends (2018).

  4. Spencer Soper: Amazon Will Consider Opening Up to 3,000 Cashierless Stores by 2021 (2018).

  5. Grewal, D., Roggeveen, A.L., Nordfält, J.: The future of retailing. J. Retail. 93(1), 1–6 (2017)

    CrossRef  Google Scholar 

  6. Priporas, C.-V., Stylos, N., Fotiadis, A.K.: Generation Z consumers’ expectations of interactions in smart retailing: a future agenda. Comput. Hum. Behav. 77, 374–381 (2017)

    CrossRef  Google Scholar 

  7. Djelassi, S., Diallo, M.F., Zielke, S.: How self-service technology experience evaluation affects waiting time and customer satisfaction? A moderated mediation model. Decis. Support Syst. (2018).

    CrossRef  Google Scholar 

  8. Grewal, D., Levy, M., Kumar, V.: Customer experience management in retailing: an organizing framework. J. Retail. (2009).

    CrossRef  Google Scholar 

  9. Kallweit, K., Spreer, P., Toporowski, W.: Why do customers use self-service information technologies in retail? The mediating effect of perceived service quality. J. Retail. Consum. Serv. (2014).

    CrossRef  Google Scholar 

  10. Poncin, I., Ben Mimoun, M.S.: The impact of “e-atmospherics” on physical stores. J. Retail. Consum. Serv. (2014).

    CrossRef  Google Scholar 

  11. Roy, S.K., Balaji, M.S., Sadeque, S., Nguyen, B., Melewar, T.C.: Constituents and consequences of smart customer experience in retailing. Technol. Forecast. Soc. Change (2017).

    CrossRef  Google Scholar 

  12. Willems, K., Smolders, A., Brengman, M., Luyten, K., Schöning, J.: The path-to-purchase is paved with digital opportunities: an inventory of shopper-oriented retail technologies. Technol. Forecast. Soc. Change (2017).

    CrossRef  Google Scholar 

  13. Lazaris, C., Vrechopoulos, A.: Human-computer vs. consumer-store interaction in a multichannel retail environment: some multidisciplinary research directions. In: Nah, F.F.-H. (ed.) HCIB 2014. LNCS, vol. 8527, pp. 339–349. Springer, Cham (2014).

    CrossRef  Google Scholar 

  14. Poncin, I., Garnier, M., Ben Mimoun, M.S., Leclercq, T.: Smart technologies and shopping experience: are gamification interfaces effective? The case of the Smartstore. Technol. Forecast. Soc. Change (2017).

    CrossRef  Google Scholar 

  15. Ajzen, I., Fishbein, M.: Understanding Attitudes and Predicting Social Behavior. Prentice Hall, ‎Englewood Cliffs (1980)

    Google Scholar 

  16. Davis, F.D.: User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. Int. J. Man-Mach. Stud. 38(3), 475–487 (1993)

    CrossRef  Google Scholar 

  17. Davis, F.D., Bagozzi, R.P., Warshaw, P.R.: User acceptance of computer technology: a comparison of two theoretical models. Manage. Sci. 35(8), 982–1003 (1989)

    CrossRef  Google Scholar 

  18. Landmark, A.D., Sjøbakk, B.: Tracking customer behaviour in fashion retail using RFID. Intl. J. Retail. Distrib. Manag. (2017).

    CrossRef  Google Scholar 

  19. Parasuraman, A., Grewal, D.: The impact of technology on the quality-value-loyalty chain: a research agenda. J. Acad. Mark. Sci. 28(1), 168–174 (2000)

    CrossRef  Google Scholar 

  20. Penttinen, E., Rinta-Kahila, T., Ronkko, M., Saarinen, T.: Triggering intention to use to actual use – empirical evidence from self-service checkout (SCO) systems. In: 2014 47th Hawaii International Conference on System Sciences (HICSS), Waikoloa, HI, 06.01.2014–09.01.2014, pp. 3347–3355. IEEE (2014).

  21. Wang, M.C.-H.: Determinants and consequences of consumer satisfaction with self-service technology in a retail setting. Manag. Serv. Qual. (2012).

    CrossRef  Google Scholar 

  22. Orel, F.D., Kara, A.: Supermarket self-checkout service quality, customer satisfaction, and loyalty: Empirical evidence from an emerging market. J. Retail. Consum. Serv. (2014).

    CrossRef  Google Scholar 

  23. Fernandes, T., Pedroso, R.: The effect of self-checkout quality on customer satisfaction and repatronage in a retail context. Serv. Bus. (2017).

    CrossRef  Google Scholar 

  24. Lecointre-Erickson, D., Daucé, B., Legohérel, P.: The influence of interactive window displays on expected shopping experience. Int. J. Retail. Distrib. Manag. (2018).

    CrossRef  Google Scholar 

  25. Lee, H.-J., Yang, K.: Interpersonal service quality, self-service technology (SST) service quality, and retail patronage. J. Retail. Consum. Serv. (2013).

    CrossRef  Google Scholar 

  26. White, A., Breazeale, M., Collier, J.E.: The effects of perceived fairness on customer responses to retailer sst push policies. J. Retail. (2012).

    CrossRef  Google Scholar 

  27. Mukherjee, A., Smith, R.J., Turri, A.M.: The smartness paradox: the moderating effect of brand quality reputation on consumers’ reactions to RFID-based smart fitting rooms. J. Bus. Res. (2018).

    CrossRef  Google Scholar 

  28. vom Brocke, J., Simons, A., Niehaves, B., Riemer, K., Plattfaut, R., Cleven, A.: Reconstructing the giant: on the importance of rigour in documenting the literature search process. In: ECIS (European Conference on Information Systems), vol. 9, pp. 2206–2217 (2009)

    Google Scholar 

  29. Cooper, H.M.: Organizing knowledge syntheses: a taxonomy of literature reviews. Knowl. Soc. 1(1), 104–126 (1988)

    Google Scholar 

  30. Lemon, K.N., Verhoef, P.C.: Understanding customer experience throughout the customer journey. J. Mark. (2016).

    CrossRef  Google Scholar 

  31. Holbrook, M.B., Hirschman, E.C.: The experiential aspects of consumption: consumer fantasies, feelings, and fun. J Consum. Res. (1982).

    CrossRef  Google Scholar 

  32. Verhoef, P.C., Lemon, K.N., Parasuraman, A., Roggeveen, A., Tsiros, M., Schlesinger, L.A.: Customer experience creation: determinants, dynamics and management strategies. J. Retail. (2009).

    CrossRef  Google Scholar 

  33. Varadarajan, R., et al.: Interactive technologies and retailing strategy: a review, conceptual framework and future research directions. J. Interact. Mark. (2010).

    CrossRef  Google Scholar 

  34. Meuter, M.L., Ostrom, A.L., Roundtree, R.I., Bitner, M.J.: Self-service technologies: understanding customer satisfaction with technology-based service encounters. J. Mark. (2000).

    CrossRef  Google Scholar 

  35. Blázquez, M.: Fashion shopping in multichannel retail: the role of technology in enhancing the customer experience. Int. J. Electron. Commerce (2014).

    CrossRef  Google Scholar 

  36. Bustamante, J.C., Rubio, N.: Measuring customer experience in physical retail environments. J. Serv. Manag. (2017).

    CrossRef  Google Scholar 

  37. Siah, J.W., Fam, S.-F.: Self-checkout service quality and purchasing intention in Malaysia. J. Fundam. Appl. Sci. 10(6S), 2669–2682 (2018)

    Google Scholar 

  38. Wang, C., Harris, J., Patterson, P.: The roles of habit, self-efficacy, and satisfaction in driving continued use of self-service technologies: a longitudinal study. J. Serv. Res. (2013).

    CrossRef  Google Scholar 

  39. Hunneman, A., Verhoef, P.C., Sloot, L.M.: The moderating role of shopping trip type in store satisfaction formation. J. Bus. Res. (2017).

    CrossRef  Google Scholar 

  40. Beck, M., Crié, D.: I virtually try it … I want it! Virtual fitting room: a tool to increase on-line and off-line exploratory behavior, patronage and purchase intentions. J. Retail. Consum. Serv. 40, 279–286 (2018)

    CrossRef  Google Scholar 

  41. Poushneh, A.: Augmented reality in retail: a trade-off between user’s control of access to personal information and augmentation quality. J. Retail. Consum. Serv. 41, 169–176 (2018)

    CrossRef  Google Scholar 

  42. Pantano, E.: Engaging consumer through the storefront: evidences from integrating interactive technologies. J. Retail. Consum. Serv. (2016).

    CrossRef  Google Scholar 

  43. Eberhardt, G.: Guten Tag, wie kann ich Ihnen behilflich sein? Über Welcome Agents im Einzelhandel (2018).

  44. Recker, J. (ed.): Scientific Research in Information Systems: a Beginner’s Guide, pp. 76–87. Springer, Heidelberg (2012).

    CrossRef  Google Scholar 

  45. Hufnagel, E.M., Conca, C.: User response data: tpotential for errors and biases. Inf. Syst. Res. 5(1), 48–73 (1994)

    CrossRef  Google Scholar 

  46. Riedl, R., Léger, P.-M.: Fundamentals of NeuroIS - Information Systems and the Brain. Springer, Heidelberg (2016).

    CrossRef  Google Scholar 

  47. Neben, T., Xiao, B.S., Lim, E., Tan, C.-W., Heinzl, A.: Measuring appeal in human computer interaction: a cognitive neuroscience-based approach. In: Davis, F.D., Riedl, R., vom Brocke, J., Léger, P.-M., Randolph, Adriane B. (eds.) Information systems and neuroscience. LNISO, vol. 10, pp. 151–159. Springer, Cham (2015).

    CrossRef  Google Scholar 

  48. Webb, N., Renshaw, T.: Eyetracking in HCI. Research Methods for Human-Computer Interaction, pp. 35–69 (2008)

    Google Scholar 

  49. Weinert, C., Maier, C., Laumer, S.: What does the skin tell us about information systems usage? A literature-based analysis of the utilization of electrodermal measurement for IS research. In: Davis, F.D., Riedl, R., vom Brocke, J., Léger, P.-M., Randolph, A.B. (eds.) Information Systems and Neuroscience. LNISO, vol. 10, pp. 65–75. Springer, Cham (2015).

    CrossRef  Google Scholar 

  50. Müller-Putz, G.R., Riedl, R., Wriessnegger, S.C.: Electroencephalography (EEG) as a research tool in the information systems discipline: foundations, measurement, and applications. Commun. Assoc. Inf. (37), 46 (2015)

    Google Scholar 

  51. Riedl, R.: Zum Erkenntnispotenzial der kognitiven Neurowissenschaften für die Wirtschaftsinformatik: Überlegungen anhand exemplarischer Anwendungen. Neuro. Psychoanal. Econom. 4(1), 32–44 (2009)

    Google Scholar 

  52. Riedl, R., Davis, F.D., Hevner, A.R.: Towards a NeuroIS Research Methodology: Intensifying the Discussion on Methods, Tools, and Measurement. JAIS (2014).

    CrossRef  Google Scholar 

  53. Vogel, J., Auinger, A., Riedl, R.: Cardiovascular, neurophysiological, and biochemical stress indicators: a short review for information systems researchers. In: Davis, F.D., Riedl, R., vom Brocke, J., Léger, P.-M., Randolph, A.B. (eds.) Information Systems and Neuroscience. LNISO, vol. 29, pp. 259–273. Springer, Cham (2019).

    CrossRef  Google Scholar 

  54. Narzt, W.: A comparison of attention estimation techniques in a public display scenario. In: Nah, F.F.-H., Tan, C.-H. (eds.) HCIBGO 2017. LNCS, vol. 10293, pp. 338–353. Springer, Cham (2017).

    CrossRef  Google Scholar 

  55. Larsen, N.M., Sigurdsson, V., Breivik, J.: The use of observational technology to study in-store behavior: consumer choice, video surveillance, and retail analytics. Behav. Anal. (2017).

    CrossRef  Google Scholar 

  56. Le, H.T., Vea, L.A.: A customer emotion recognition through facial expression using Kinect Sensors v1 and v2. In: Unknown (ed.) Proceedings of the 10th International Conference on Ubiquitous Information Management and Communication - IMCOM 2016. The 10th International Conference, Danang, Viet Nam, 04.01.2016–06.01.2016, pp. 1–7. ACM Press, New York (2016).

  57. Noldus Information Technology: FaceReader (2019).

Download references


The present work was conducted within the training network project PERFORM funded by the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 76539. This study reflects only the authors’ view, the EU Research Executive Agency is not responsible for any use that may be made of the information it contains.

Author information

Authors and Affiliations


Corresponding authors

Correspondence to Gabriele Obermeier or Andreas Auinger .

Editor information

Editors and Affiliations

Rights and permissions

Open Access This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

Reprints and Permissions

Copyright information

© 2019 The Author(s)

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Obermeier, G., Auinger, A. (2019). Human-Computer Interaction in Physical Retail Environments and the Impact on Customer Experience: Systematic Literature Review and Research Agenda. In: Nah, F.FH., Siau, K. (eds) HCI in Business, Government and Organizations. eCommerce and Consumer Behavior. HCII 2019. Lecture Notes in Computer Science(), vol 11588. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-22334-2

  • Online ISBN: 978-3-030-22335-9

  • eBook Packages: Computer ScienceComputer Science (R0)