Keywords

1 Introduction

The concept of ‘omnichannel retail’ is centered around providing customers with a holistic shopping experience [1]. It has been established as the next step to digital retail [2] that makes it important for the retailers to stay relevant in the business. Several retailers have already started with this transition to omnichannel and have benefited from it [3]. Omnichannel retail has also been suggested as one of the tools to battle the ‘retail apocalypse’ [4], and consequently meet the expectations of a progressive retail customer. Along with this, research around digital and omnichannel retail has become more important as a consequence of COVID-19 where traditional retail environments are forced to adopt newer models and technologies [5].

An omnichannel customer expects their shopping journey to be seamless and attributed with utilitarian and hedonic values, while they evaluate their shopping experience by interacting with the shopping environment [1]. One of the key attributes of an omnichannel retail environment is the optimal deployment of novel digital technologies like digital realities, smartphones and recommender systems to enhance the customer’s shopping journey [6]. Among these technologies, Mixed Reality (MR) has emerged as one of the key technology megatrends that has the potential to revolutionize the retail sector in the next decade [7]. MR has a unique capability to bridge the gap between online and offline environments which fits right into the concept of omnichannel retail where multiple channels are integrated into a single seamless customer journey. Furthermore, a shopping assistant system that leverages the qualities of MR technology to tackle customers' needs and improve their experience is one of the preeminent ways to row through the transition to omnichannel retail [8].

Design Science Research (DSR) methodology can be used to develop such systems, as it provides a rigid framework to produce physical artifacts that can solve real-world organizational problems [9,10,11]. This is an iterative approach where the evaluation of the physical artifact is an important step to optimize the next iteration. Traditionally, Technology Acceptance Model (TAM) [12] has been used in this regard to capturing user perception over innovative technologies. While the literature advances, a number of authors have manifested TAM to fit better into their particular context, for e.g., the authors in [13] added the hedonic element ‘enjoyment’ to their studies. Other constructs like user’s privacy and trust have also been studied to influence the perception of retail customers as they emerge as a topic of concern in the current literature [14]. These factors can influence the customer’s overall perception of retail technologies which can affect different shopping outcomes for retailers and their businesses.

Although the present literature brings out a lot of different advantages of MR, and several application designs towards retail [15], more research is required towards the understanding of user’s perception towards the technology. This is the targeted research gap, which also builds upon the research framework mentioned in [16]. Towards this, we firstly use DSR to design an omnichannel retail shopping assistant system using ‘optical see-through mixed reality’ and Microsoft (MS) HoloLens and Microsoft HoloLens 2 as two different hardware archetypes. We use the present industry and academic standards to develop a pseudo-optimal system, hence contributing a physical instantiation as the research outcome.

Furthermore, we captured user perception towards the designed artifact in a laboratory study with 29 participants. An extended TAM was used to study users' behavioral intentions. The results reveal the relationship between the different shopping constructs. These results along with the qualitative comments from the participants during the study are used to extract research, development and deployment implications. These implications aim to contribute towards a better understanding of the general user perception towards MR-based retail systems and suggest improvements over the next iteration of the prototype. Hence, we add to the current literature of MR systems in retail environments.

2 Background and Previous Work

2.1 Optical-See Through Mixed Reality

The ‘Virtuality continuum’ defined by Paul Milgram [17] categorizes optical see-through mixed reality (OSTMR) as class 3 displays. These devices are “head-mounted displays equipped with a see-through capability, with which computer-generated graphics can be optically superimposed” [17, p. 3]. As there is limited literature on the deployment of OSTMR in retail, we first reviewed some literature from similar technologies. For example, Authors in [18] and [15] deployed smartphone-based MR applications that aim to enhance the in-store shopping experience. However, smartphones are limited in terms of providing a natural posture for interaction and communicating tangible characteristics of a product to the customer. Other devices like Oculus [19], which uses the concept of fully immersive virtual reality (VR) also has been studied in this regard. Fully immersive VR poses a challenge for customers as it occludes the vision of a user and hence, obstructs their primary tasks in a natural environment. Also, non-immersive technologies like display screen monitors have been used in this regard to develop physical artifacts for an enhanced customer journey. A 2D display like a monitor screen or a smartphone is unable to project an organic environment even with touch and sensor-based input methods. These technologies have proven their significance for a long time but still is abstain from providing hedonism and natural interactions [20]. OSTMR complements these technologies by having the capability of not completely occluding the vision of a user during the use, and creating a pseudo immersive environment where digital and physical objects co-exist and interact with each other. This makes the experience more tangible, interactive, and exciting for customers as compared to other display and interaction technologies. MS HoloLens executes this task using a holographic display that projects ‘holograms’, which are digital objects rendered into the real world. These holograms are enhanced by sound and light and can be interacted with, using interaction techniques provided by the device [21].

2.2 MR Shopping Assistant System

A contemporary customer has high expectations in terms of their shopping experience due to the development of technologies. Retailers need to address these needs and expectations to stay relevant in the business [22]. This can be done by providing personalized and tailored assistance with digital shopping assistants [7]. Personal MR devices can provide this assistance putting the customers in control of their shopping journey and helping them towards comparing products, finding alternatives, and feeling more confident in their decision-making at the purchase. Previous authors have used OSTMR to develop such systems in different settings. Recent literature shows examples such as [23] where the authors used this technology to develop an in-store recommender system that can provide tailored recommendations to customers. Other examples brought out features like product detection [24] and Natural User Interaction (NUI) [25] that can be used in an optimal MR shopping assistant design. Authors in [26] suggest the use of product information and reviews in a shopping assistant system while mentioning the importance of hardware design. Authors in [27] point out the significance of the ‘buy’ button in a shopping interface. Collectively, these shopping elements and MR features can help design a pseudo-optimal MR shopping assistant system.

2.3 Customer Experience and Hypotheses Development

Customer experience is considered to be the center of omnichannel retail business models [2]. It has been defined to be “holistic in nature and involves the customer’s cognitive, affective, emotional, social and physical responses to the retailer” [28, p. 70]. Thus, it is absolutely important to understand customers' perceptions and responses in order to create digital solutions towards enhancing their shopping journey [1]. The customer experience can consist of a plethora of constructs that can be based on subjective and objective attributes. The current work does not aim to report an exhaustive account of all the customer experience constructs but works on a set of constructs that have been either studied extensively in academic literature or are relevant in the current age. One of the most used models to test the usability, and capture the user perception towards a new technology-based system is TAM (Davis 1985), which predicts the user’s Intention to use (ITU) a system using ‘Perceived usability’ (PU), and ‘Perceived ease of use’ (PEOU). PU and PEOU are collectively used to define ‘technology adoption’ in the current work. Along with the utilitarian assistance provided by the desired solution, hedonic motivations such as fun, pleasure, and enjoyability [29] are crucial factors influencing customers’ shopping experience. ‘Enjoyability’ has been used as an added parameter of technology adoption [30, 31], even especially for MR [32], where the authors bring out the importance of the construct and its positive effect on factors like purchase intention and the attitude of the customers in a shopping journey. Developing on these findings, we propose the first hypothesis:

  • H1: Perceived usefulness (H1a), perceived ease of use (H1b), and perceived enjoyment (H1c) have a significant effect on the intention to use the MR shopping assistant in omnichannel retail.

‘Security beliefs’ consisting of ‘Privacy concerns’ and ‘Trust’ are adapted from [26]. The authors argued that trust and privacy concerns play an important role in the general perception of the shopping environment and whether customers will use the technologies or not. The authors used two shopping assistance systems that leverage either bar-code scanner or radio frequency identification (RFID) reader as the hardware design, while we want to study the effects in MR technology. The authors in [14] state that security concerns have risen with technologies like MR, which can affect the customer perception in a retail environment. Hence, we propose the second hypothesis as:

  • H2: Privacy concerns (H2a) and trust (H2b) have a significant effect on the intention to use the MR shopping assistant in omnichannel retail.

Shopping outcomes comprise of ‘Convenience’, ‘Word-of-mouth’, ‘Attitude towards a retailer’, and ‘Customer service quality’ [26]. These outcomes are some of the widely studied constructs in context to customer experience and perception [33,34,35]. These constructs have been mentioned in the literature to have a great impact on the retailer’s businesses, and hence are important to be researched. As digital technology is an important part of an omnichannel retail environment, we believe that the intention to use the system can shape these outcomes. Hence, we propose the following hypothesis.

  • H3: The intention to use the MR shopping assistant in omnichannel retail has a significant effect on the shopping outcomes: Convenience (H3a), Word-of-mouth (H3b), Customer service quality (H3c) and Attitude towards a retailer (H3d).

3 MR Shopping Assistant System

The designed system is summarized below using the eight components of the information systems design science principles mentioned in [9].

Purpose and Scope:

The mixed-reality digital shopping assistant application was designed to provide customers with an exciting and helpful shopping journey.

Constructs:

Technology adoption (perceived usefulness, perceived ease-of-use), intention to use, enjoyment, security beliefs (privacy concerns, trust), shopping outcomes (convenience, word-of-mouth, customer service quality, attitude towards a retailer).

Principle of Form and Function:

The blueprint of the IT artifact involved hardware and software design:

Hardware Design (MS HoloLens and MS HoloLens 2):

The produced artifact in the form of a digital shopping assistant is deployed over two different OSTMR devices of the same family: MS HoloLens and HoloLens 2. The first generation of MS HoloLens introduced a whole new ecosystem of immersive technology devices and was projected as the ‘The future of augmented reality’ [21]. Despite its success as a developer prototype, certain limitations of the hardware were reported like the narrow field of view, the complexity of interaction methods, ergonomics, etc. The second generation, HoloLens 2 brings several improvements for the first-generation device such as a dedicated DNN core, wider field of view, improved ergonomics, articulated hand tracking, and eye gaze tracking [36]. Both devices aim to deploy a multimodal NUI based on hand gestures and voice interaction, but HoloLens 2 claims to have a more natural interaction as the digital holograms can be ‘touched’ like physical objects as compared to the ‘gaze and commit’ scheme of HoloLens. However, the shortcomings of the earlier device can be compensated by using the HoloLens Clicker, which is a handheld click-based interaction device, which reduces the physical complexity of the HoloLens interaction schema. Arguably, this reduction in the physical complexity of interaction can take over the reward of naturalness in HoloLens 2. This will lead to a similar perception for both of the devices which also implies higher scalability of the designed interface.

Software Design (Information, Recommendation, Reviews, and a Buy Button).

The system largely builds over the requirements developed by the authors in [23]. We deployed image recognition using Vuforia Engine [37] that enables the system to detect the product of interest that is brought on to its field of view (FOV). Once the device recognizes the product, a 3D digital interface is placed around the object as shown in Fig. 1 (left and right).

Fig. 1.
figure 1

Application design: HoloLens 2 (left), HoloLens (right); Participant interacting with the application (center)

Full size image

The interactive user interface is designed with the help of standardized tools: Mixed Reality Toolkit (MRTK and MRTK2) for HoloLens1 and HoloLens 2 [38]. The UI contains four major elements: ‘Information,’ ‘Recommendations,’ ‘Reviews,’ and ‘Buy’ as shown in Fig. 1. These elements are represented as 3D buttons that are anchored on to the product. Information and reviews, as a shopping assistant element has been adapted from [26, 39] where the combination of these two elements in the artifact was preferred by the customers in comparison to the absence of them or presence of only one of the elements. Mining data from different channels to provide a customized service such as product recommendations on ubiquitous devices, such as the HoloLens could create multiple benefits in the retail ecosystem [40]. For retail customers, the product recommendations boost the efficiency in finding preferential products, provide more confidence in making a purchase decision, and give a potential chance to discover something new. These assistance items aim to reduce information overload and enhance decision-making [41] as the customer is more confident in the buying process. Furthermore, it was necessary to introduce a one-click checkout UI item [42], following web-based shopping interfaces. This is addressed with the ‘Buy’ button in the UI, which eases the customer’s path-to-purchase and target the customer’s need to buy the product immediately, raising customer satisfaction, solving the ‘crisis of immediacy’ [22], and integrating the customer journey into one channel/touchpoint. Both devices are rigged with a voice recognition system which can be used by the user to disable or enable the holograms attached to the product by saying ‘start’ and ‘stop’ respectively.

Artifact Mutability:

The artifact designs were developed toward scalability. Although during the tests, the application content was static, a retailer’s databases could be linked to the artifacts. By having access to real-time data, the artifacts could adjust to the dynamic retail environment.

Testable Propositions:

Nine prepositions were constructed based on the present literature (see Sect. 2).

Justificatory Knowledge:

Drawing on existing literature, generalizations were constructed from patterns observed in academic literature and industry trends.

Principles of Implementation:

Several recommendations for implementing the learnings in future research and iterations are provided in the form of implications.

Expository Instantiation:

The MR shopping artifacts were implemented in a simulated omnichannel retail environment.

4 Experiment Setup

The study was conducted with 29 participants who tried the MR application on both devices (HoloLens, HoloLens 2). The participants were recruited using e-mail and instant message-based invitations. The order of the devices was randomized and distributed evenly among the population. Sixteen (55%) of the participants were male while the other 45% were female. The participants were from ten different nationalities, however, 17 (59%) were German. The mean age of the participants was 28 years with a standard deviation of 4.8. Eighteen (62%) of the participants answered “yes” to the question “Have you had any experience with Mixed-Reality before this study?”, and hence had prior experience with MS HoloLens or similar immersive environments. The participation was voluntary, and no financial compensation was provided.

An omnichannel retail environment was simulated in a laboratory with a hypothetical retailer ‘AWS’. The setting consisted of two different categories of products that are considered to diversify the product assortment in the setting: Search products (a pack of milk, a computer monitor, a pair of sneakers, a package of soft-drinks, a pack of coffee beans) and experience products (i.e., a box of chocolate, a tech magazine, a bottle of rum, a video game disc, a 6-pack of beer).

The setup was made to look casual, and less like a traditional brick-and-mortar store. Written consent was obtained from the participants which was followed by a short introduction to the application and the environment. Then, the participants were assigned the task of ‘general browsing’ where they were asked to browse and buy the available products as they want. This was done using verbal and written instructions. No real money was involved in the buying process and the task was more oriented towards testing the functionalities of the application and experiencing the shopping environment. General browsing was chosen over goal-oriented because the goal of the study focused more on a participant’s perception and opinion about technology and less on the efficiency of the application during the shopping journey. The participants were then asked to complete a questionnaire that consisted of 34 items. They rated their responses on a 7-point Likert Scale. Scales were adopted from [26] for technology adoption, intention to use, security beliefs, and customer shopping outcomes. Items in the scale for enjoyability were adopted from [13, 32]. The complete questionnaire is attached in the appendix Table A1. The study took less than 60 min per participant with an exposure of approximately 30 min to the MR devices ensuring that there is no simulation sickness to deviate the user’s opinion towards the technology.

5 Results

The scales’ reliability was tested, and the cronbach’s alpha was greater than 0.90 with both the devices, hence making the scale reliable. Table 1 summarizes the participants’ response towards the technology, with both the hardware devices. Paired t-test was used to compare the two hardware and the results show a significant difference between the perceived privacy concerns in HoloLens (HL1) and HoloLens 2 (HL2). As a higher value with privacy concerns means fewer privacy concerns (see Table A1), it can be interpreted that the participants felt more comfortable with HoloLens 2 towards their privacy.

Table 1. Mean and Standard deviation (SD) for HoloLens1 and HoloLens 2
Full size table

5.1 Regression Analysis

The Hypotheses proposed in Sect. 2 were tested using linear regression analysis. The regression coefficient was calculated along with the significance (p-values) of the relationships. Figure 2 presents the results from the analysis. The hypotheses are tested with the criteria that if p < 0.05, the null hypothesis can be rejected and the proposed hypothesis is supported. The results show that perceived usefulness, perceived ease of use and enjoyment has a significant effect on the intention to use, thus supporting H1a-c. The results also show a significant effect of privacy concerns and trust on intention to use with both hardware even though privacy concerns have a higher effect in the case of HoloLens 2. Also, intention to use is seen to have a significant effect on all the shopping outcomes (convenience, word-of-mouth, customer service quality and attitude towards a retailer). Hence, H2a-b and H3a-d are fully supported.

Fig. 2.
figure 2

Hypotheses testing using simple linear regression, Note: * p < .05, ** p < .01, *** p < .001

Full size image

5.2 Qualitative Comments from the Participants

During the study, the participants were allowed to make comments and ask questions about the use of the prototype. As HoloLens and HoloLens 2 are not designed in the current age to be used by naïve customers, participants mentioned a number of ergonomic issues with the hardware. This was observed with both devices in terms of heaviness and general comfort. The participants also mentioned interaction and visualization problems, for e.g., the clipping of 3D objects in the case of HoloLens. The users responded positively towards the overall usability of the interface and made some suggestions for the next iteration. This includes changing the ‘one-touch’ buying mechanism to a more secure virtual cart-based process that gives them a chance to review the products before the payment. Another suggestion was made to add meta-information elements that are either unique to an MR experience, or that combine online e-commerce and the traditional brick and mortar retail environment.

6 Discussion and Implications

The quantitative results along with the qualitative comments from the participants can have several implications for future research and development. The quantitative results in Table 1 show a positive reaction towards the MR shopping assistant system, while the quantitative comments suggest improvement over the next iteration of the prototype. The implications from these results are as follows:

Research Implications

  • As participants made valuable suggestions and comments during the quantitative study, we motivate researchers to follow a mixed-method research method that aims to qualitatively triangulate the quantitative findings to confirm and expand the knowledge and causes of the proposed hypothesis [43].

  • Following simple linear regression, multiple linear regression and mediation analysis should be used to better model the omnichannel retail customer experience in MR environments.

  • With HoloLens 2, participants showed fewer concerns towards privacy issues which calls for further privacy research with mixed reality environments.

Development Implications

  • Virtual shopping carts should be used in an MR shopping assistance system to facilitate shopping for customers.

  • Developers should highlight the novelty of MR environments by integrating shopping assistant elements that provide unique value to the customers. This can be done by integrating online and offline elements into a single application.

  • An MR shopping assistant application should focus on the efficiency of interactions and the effects of visualization. This can be done by either iterating the prototypes repeatedly using subjective evaluations or providing an option to tailor the user interface according to the customer’s needs.

Deployment Implications

  • MR shopping assistant systems are constrained by a number of constructs. These include technology adoption constructs (PU, PEOU), enjoyability privacy concerns and trust among others that can influence the intention to use the technology, which further affects a customer’s perceived convenience, word-of-mouth towards the retailers, perceived service quality and attitude towards the retailer.

  • Even though the results suggest a positive perception of MR technology in retail, as in Table 1, the retailers should be mindful of the deployment constraints and customize their omnichannel solutions according to their business needs and customer perception.

7 Conclusion

In the current age where the retail sector is forced to transition into an omnichannel paradigm due to industry trends and environmental factors like COVID-19, it is important for retailers to deploy innovative retail solutions in their businesses to enhance their customer’s shopping experience. In the current research, we first designed an OSTMR shopping assistant system using MS HoloLens and HoloLens 2 as the hardware archetypes. We integrated product information, reviews, recommendations, and a buy button into a 3D interface using the DSR approach. The evaluation of the system in a laboratory study suggests a positive perception of a user towards the technology, while several research, development and deployment implications are extracted from the quantitative results and the qualitative comments.