Keywords

1 Introduction

When collaborating humans engage in a range of rituals, which at first glance may appear insignificant. However, those ancillary activities set the context for the interaction and require effort from participants, but are rarely acknowledged as work. Consider a group brainstorming: would you, as a participant, pay attention to the distribution of blank post-its at the beginning? Consider a lively group conversation: would you observe the shaking hands ritual when a person joins the setting? Trivial acts like distributing post-its or shaking hands, seem out of scope – the participants’ focus is on the actual work. Nevertheless, ancillary activities are key: brainstorming needs post-its, or an equivalent, and greeting a person in many cultures involves a handshake.

Especially when introducing and greeting another person, humans engage in ancillary activities, which seem necessary but are unrelated to the content or topic of the conversation. Handshakes, smiles, and mutual gazing follow each other in a synchronized order and the participants execute them without explicit effort or conscious focus. However, if one does not engage in those acts, others might take it as an offense or antisocial behavior. In such moments, when something does not work as expected, humans notice that those ancillary activities are a crucial part of the ‘real work.’ If one forgets people’s names when greeting them, it is as if there were not enough post-its for everyone during brainstorming. It seems that some performances, though apparently superfluous, are essential for subsequent interaction between humans.

Goffman frames those routine, socially expected and formulaic performances as interaction rituals [27]. He provides evidence that those rituals solidify the feelings of regard and respect for oneself and other participants, thus setting the stage for effective interaction [70]. By describing the extraordinary richness of interaction rituals in many situations, like introductions or greetings, Goffman draws a picture of ancillary activities as ordered and rich performances which are integral to the social interaction rather than simply accompanying it [27, 40]. He moves the ancillary activities to the center of attention and, by using ritual as metaphor, adds a touch of sacred, spiritual meaning to social interaction between individuals [40]. We follow up on Goffman’s concept of an interaction ritual as distinct actions, which order an interaction and allow the participants to present themselves in a desired manner. In this article, we focus on the discrete rituals involved in introduction sequences.

Introductions do not involve only humans. Humans may introduce new material, new collaboration tools, or new software when interacting with others. However, only a handful of researchers analyze introduction rituals related to materials or technology in a face-to-face interaction. Even fewer app and technology developers evaluate their designs from this point of view. This holds true in particular for collaborative systems: studies highlight the advantages, usage patterns, and appropriation moves, but the act of introducing the application into the ongoing interaction remains obfuscated. It seems like collaborative applications were simply present during collaboration – anything else remains an ancillary activity. Consequently, we have a limited understanding of what rituals humans employ when a collaborative application is being introduced into an ongoing face-to-face interaction. The main objective of this study is to describe introduction rituals, i.e., singled out, routine and formulaic activities of individuals participating in a collaborative scenario, which feature the introduction of an IT-based collaborative application. Additionally, the study aims to compare the identified rituals with introduction rituals described in the literature or observed in non-IT settings to give the reader a better sense of how they are performed. In detail the study explores the relationship between rituals emerging around a collaborative application and those involving interaction with humans, where the performances provide a way to show respect to someone. Making rituals explicit and accessible can help engineers to embrace and implement them in the design of collaborative software.

Based on the analysis of video recordings from financial advisory services, this study identifies introduction rituals that occur when the advisor introduces a collaborative application into an ongoing advisory service. Comparison to the act of introducing a common material artefact (a brochure) reveals that introducing the collaborative application makes use of a wider range of interaction rituals. Comparison to the literature on introduction suggests that when introducing computer software applications, the participants tend to employ rituals otherwise typical for human introductions and greetings. Introduction of a brochure does not exhibit such tendencies. The study employs multiple perspectives to provide a rich description of introduction rituals and identify which features of the collaborative application afford those rituals.

The study builds upon previous discourses in research and practice. It adds to the discourse on interaction with IT in collaborative settings by supporting the computers-are-social-actors argument: participants embrace a software application from the very beginning, from the act of introducing it, as a social actor [55, 66] and produce performances that are normally used to show respect to other humans. This complements previous research on introducing and initiating use of a computer in an institutional setting, e.g., between a doctor and a patient [60]. It also adds to the conversation analysis efforts to understand the role of materials in coordinating communication between humans [16]. It should sensitize designers to the importance of introductions and provide them with stimuli and patterns to think about how users might proceed while using their application during collaboration. In addition, practitioners such as frontline employees at banks and other institutions may benefit from a deep analysis of micro-level behaviors and better understand how those behaviors affect the client. Overall, the study provides insights that may contribute towards software design and behavioral research.

2 Related Work

This paper intends to explicate introduction rituals involving a collaborative application and to compare those rituals against the backdrop of other introduction rituals just mentioned. This section provides relevant background starting with the most traditional form of introduction rituals involving an individual, and then introduction rituals involving materials such as brochures or sheets of paper (being the most widely used collaborative resources), and finally the scarce literature covering the introduction of IT. Whereas the literature acknowledges the ritual nature of introduction between humans, material and IT-focused performances are presented as material behaviors, even though they order the sequence of an interaction and let the humans keep up appearances, thus matching the characteristics of rituals.

2.1 Introducing a Person

Conversation analysis (CA) studies established a solid basis of knowledge about how people initiate interaction with other humans [41, 63, 68]. The literature differentiates between two types of introductions: self-initiated introduction (when a person introduces oneself) and other-initiated introductions [12, 62]. Other-initiated introductions are situations in which a third person, such as a mediator, introduces a newcomer to a pre-present person or pre-existing group of people, i.e., the individuals who had interacted with the mediator before the newcomer arrived [62]. CA studies identify a range of ritualized verbal and non-verbal behaviors which accompany the introduction of a person [22, 49, 51, 52, 56, 64].

When introducing a person, humans use a range of non-verbal and verbal cues. Whereas cultural differences may occur, here we consider introductions among Europeans and Americans. Imagine a situation that might happen at a conference. Ann and Marc are enjoying a conversation during a coffee break, while Bob, a third colleague, is getting closer to them. Bob is approaching while signalizing an intention to join the conversation with gaze and movement [64]. Ann knows Bob and steps in the role of the mediator. According to CA studies, the interaction is likely to unfold as follows: Ann changes her position such that the three will form a circle [14, 41]; Marc adapts his position as well [62]. Ann makes a deictic reference to Bob (like “This is Bob”) including details relevant to the context (job, position, name, etc.) [61, 63]. The deictic utterances involve specific gestures: Ann gestures toward Bob when introducing him [62]. Marc follows the indication and looks at Bob. Thereafter, Marc is likely to initiate interaction with Bob using a greeting (“Hello”, “Good morning”) and an adequate bodily reaction (a handshake or a short wave) [56, 62]. He might smile [22] and adapt a welcoming posture [41] – mutual gazing and palms turned open [22]. A range of verbal behaviors comes thereafter, including introduction-specific assessments (“nice to meet you”), opening-specific utterances (“how are you?”), and repetition of the name of the introduced person (Marc saying “Hi Bob”), as well as emotional semi-language (“oh”, “ah”, “well”) [62]. The sequence takes few moments and the performances are well coordinated, even among strangers [62, 63]. Introductions have a ritual character similar to theatrical performances – all participants seem to know what to do, such that the interaction unveils smoothly [22, 29]. Furthermore, the participants behave in a respectful way and produce multiple signs of respect (positions, postures, emotional or polite utterances), which underscores the fact that rituals reinforce social order and signalize the mutual recognition between the parties [27, 40, 70]. The ritual relies not only on the mere occurrence of particular behaviors, but also on the relations between them: their sequence, mutuality, and the links between the verbal and non-verbal conduct [51, 52].

The configuration involving a mediator (Ann), a newcomer (Bob) and a pre-present participant (Marc) resemble what happens when a computer enters the stage in an advisory service. The advisor, as the one who knows the system and hosts the interaction, takes on the role of the mediator and introduces the collaborative application to the pre-present client. However, it remains unclear whether and to what extent insights about the introduction rituals related to a person can be transferred to a situation where a material, – a collaborative application or any form of interactive IT – gets introduced.

2.2 Introducing a Material Artefact

Whereas introduction among humans has been studied for decades, studies on how individuals introduce new material into an ongoing interaction arrived much later [48, 52]. Only a few articles consider the introduction and use of material artefacts in informal collaboration [53]. The primary focus lies on institutional settings ranging from academic supervision [54, 74] to performance appraisal interviews [48] to doctor-patient encounters [11] to the financial advisory services at a bank [16]. Given that paper (brochures, forms, notepads, etc.) remains the most widely spread material in institutional collaboration, most insights refer to the use of paper [16, 50, 74]. The analyses encompass the use of material in various situations as a whole and only rarely address a particular phase of interaction like beginning or closing [9, 52]. Instead, most CA studies relate the material rituals to conversational processes like turn taking or activity shifts [28, 31, 52, 54, 74]. This study takes a different path and focuses on the various verbal acts that accompany the initial use of an artefact.

Taking together the insights and observations from previous literature, one can identify some verbal and non-verbal behaviors typical for initial use of material artefacts. Given the fact that material usage in student counselling has been extensively researched [30, 73, 74], we explore that example here. There is solid evidence that similar behaviors may occur in financial services [16] or in doctor-patient encounters [11]. Consider a situation involving Barbara, a university teacher, and a student, Mike, sitting at a table in a university office and discussing Mike’s seminar thesis. Barbara has prepared by putting away documents that cluttered the table beforehand [30, 74]. While Mike explains what he has done so far, Barbara listens, first keeping a pen in her hand, and then leafing through a pile of documents to her left, while encouraging Mike to talk further [74]. She takes two pieces of paper and organizes the remaining ones back into the pile [74]; she places the two sheets of paper in the middle of the table, next to each other [74]. Mike stops talking and Barbara says: “Okay, I get it. Well, let’s take this first; you can read it, right?” while pointing to one of the documents. “Then we will go through that one. Maybe I remember it” – she continues while pointing to the other document. Mike nods and responds “mhm”; they continue while looking at the documents. This short scenario comprises typical behaviors that characterize the use of material in institutional collaboration [11, 16, 30, 73,74,75]. In particular, documents – even if central to the ongoing activity – enter the stage without explicit introduction. Instead, they are used more as an “excuse” to move to the next activity. Rituals involve ordering the documents (in the pile and on the table), as well as keeping up the ongoing interaction despite the act of introducing a material.

2.3 Introducing a Machine

The introduction of IT into an ongoing face-to-face interaction attracted researchers’ attention much later. The research splits into two streams: first, studies oriented around the introduction of “traditional” desktop computer [7, 20, 59, 60, 76] and second, those dealing with human-robot interaction [21, 25]. There is little overlap between the two streams: On the one hand, studies on the introduction of desktop computers appear in healthcare informatics [60, 76], studies of IT use in public domain [2, 45], or IT-oriented workplace studies [46]. On the other hand, studies from human-robot interaction and from robotics attend to the initial interactions between humans and robots in the process of developing new technologies and applications [65]. Since robots are a new phenomenon, researchers are still in the process of exploring and designing the initial interactions – rituals still need to emerge. Therefore, we focus on introduction rituals involving collaborative applications using state-of-the-art interfaces.

Computers entered doctors’ offices and public agencies years ago. Recall a typical interaction with a doctor (or a frontline employee, etc.). Let’s call her Eve. Perhaps, Eve has a monitor standing on her desk, the display turned toward her [2, 60]. There is also a mouse and a keyboard on the table [58, 60]. Eve interacts with her conversation partner, but from time to time, she turns towards the computer; she also types things using the keyboard or uses the mouse [60, 76]. The computer remains Eve’s “private” tool during the interaction; the patient/client may see some things on the screen, but they remain outside the scope of the conversation [59]. This is so until Eve turns the screen and presents an interesting artefact shown there, let’s say, an X-ray image. She points to the display, says “This is your leg, you see – there is your injury from the past” and uses her pen to pinpoint a specific area in the image [2]. Eve and her conversation partner move closer to the screen and look at the picture, then at each other [60]. Eve continues talking, while the other party nods and acknowledges with “uh huh” [19]. Situations like occur millions of times every day. The desktop computer does not need an introduction: it is already there, on the table [1, 13, 57]. Nevertheless, Eve employs deictic gestures and words when using the computer for the first time as a collaborative resource. The rituals she engages in involve deictic utterances and rotation of the screen analogous to the introduction of a leaflet as described before.

Given the framing of computers as a collaborative resource, this behavior seems adequate. However, it contrasts with an alternative popular view on computers in social interaction: “computers-are-social-actors” (CASA) [55, 66]. Research following this paradigm claims and provides evidence that humans tend to treat technology like they would treat other humans, thus applying the heuristics and behavior schemata used for humans, to computers. For instance, humans may talk to their computers or assume human intelligence behind a computer’s actions. During the introduction, one could expect behaviors akin to those of Marc, Ann and Bob rather than those of Barbara and Mike. But the literature on introducing computers into collaboration does not analyse the introduction sequences in terms of the CASA paradigm. The scarce evidence includes no indication that human participants treat computers in any special way.

3 Designing for Introduction Rituals

3.1 Context: Status Quo in IT-Supported Advisory Services

An advisory service is a collaborative encounter between a professional advisor and an advisee [23, 38]. The participants differ in terms of their knowledge, institutional identity and interaction rights. The client knows about hisFootnote 1 problems in the initial situation, which can be solved with the advisor’s expertise on the potential solutions [39]. Advisors are in charge of distributing interaction rights and dominate the encounter in verbal [17, 32] and material [17, 74] terms. It is the advisor who distributes brochures and documents across the table and invites the advisee to contribute [43]. In the current study, we focus on mortgage advisory services of a Swiss local bank.

The Swiss real estate market depends on mortgage loans. Clients attend to advisory services if they fulfill (or they think they fulfill) the minimum requirements, e.g., 20% of house price available, or they identified a property to buy [10]. During an advisory service, the advisor first learns about the advisee’s wishes and situation, then she explains the available products and makes a recommendation [38]. In a traditional mortgage advisory service, the participants first consider the chosen property, and then they assess the advisee’s financial capability and eligibility for a mortgage. If those are met, the advisor describes the possible range of choices (fix-rate and flexible mortgage, with and without amortization) and prepares a composition that fits advisee’s needs.

Rituals bind the various elements of an advisory service together, such that the participants, who never met before, establish a coherent encounter [8]. Many rituals relate to first impressions and the management of expectations [27]: a bank advisor dresses in a manner signaling her role [26, 36], positions documents in an orderly manner [16], manages mutual eye gaze and shows interest in advisee’s spoken words and body language [42]. However, introducing a collaborative application is not an established ritual. Research shows that advisors and advisees try to embrace a new system ad-hoc and establish clumsy interaction protocols [42]. As a consequence, more and longer pauses occur than in an conventional encounter [17]. Overall, the literature suggests that initiating the use of a collaborative IT system during an advisory service yields unintended interruptions and destroys key rituals.

We propose the following interpretation of the problem: The systems presented in previous research neither fit the typical rituals of advisory services (such as advisor presenting oneself with a business card [16], advisor introducing the central topic with a drawing or leaflet [16], or advisor showing knowledge about the property to be supported with the mortgage and its location [16]), nor do they explicitly afford an alternative ritual that could accompany the introduction of the collaborative application. As a consequence, the idea of an advisory encounter embodied in routines, roles and expectations falls apart and leaves the participants in an improvisation mode.

To address this problem, we launched a project to develop a system aligned with the conventional advisors’ practices. Since those practices rely on paper, pen and brochures, the designed system handles physical interaction. On the other hand, to enhance the interaction, the system allows, e.g., for dynamic adaptation of graphics, which is impossible with “normal” paper. An article, which describes the system in more detail, is under review. Herein we focus on the initial interaction with the system and thus, review only the elements used to initiate the application as a collaborative resource.

3.2 Technology: Mixed Reality for Physical Interaction

Mixed reality (MR) research has long studied integration of embodied and physical interaction with digital processing. It addresses the topic of digital and material reality in a way that allows for full use of space and body during interaction with digital content [18], thus making it possible to support specific rituals. MR embraces a set of research directions. Augmented reality (AR) studies how to overlay physical spaces with computer-generated content [3, 5]; spatial AR uses projection to put the content directly into the environment of the user [6, 15], however it requires powerful and high-fidelity overhead projectors and affordable 4 K beamers, which arrived on the market only three years ago. Organic user interfaces focus on projecting content onto non-planar spaces that users can interact with through bending, folding, and manipulating the form [33, 34]. Tangible user interfaces focus on enabling interaction, manipulation, and collaboration [69] with digital content through physical objects and space [35, 79]. Pen-and-paper user interfaces try to bridge the gap between digital interaction and paper, which remains an essential tool in many human activities [47, 71, 72]. All those areas of research share an interest in enhancing the physical world with digital content and functionalities to enable natural interaction with objects, avatars and other people [4]. The subsequent section explains how MR-inspired features were leveraged to enable introduction of the collaborative application during a mortgage advisory service.

3.3 Design: Mixed Reality for Introduction Rituals in Advisory Services

Observations and preliminary interviews with the advisors revealed that a critical point during the beginning of an advisory service is the first mention of the property. When switching from opening small talk to the core of the service, advisors often address the property in various ways [16]: an advisor demonstrates her knowledge about the specific neighborhood to give the impression of an interested and informed professional; another advisor asks questions about the private motivations behind the purchase to better understand the client’s emotions; and other advisors simply engage in a courteous discussion concerning the property to establish rapport with the client. So far, the advisors supported this part of the conversation with no material or, at most, with printouts of basic data about the house, such as an advertisement. This contributed little to the content of the conversation. Those practices inspired the design team: on the one hand, there was potential to make the conversation more informative by introducing additional information resources; on the other hand, it offered an opportunity to introduce the IT as a collaborative application before the participants move on to hard math tasks like calculating the mortgage rate and interest. Taking into account the possibilities of mixed reality, the designers proposed a house token & map design.

The house token & map builds on and extends mortgage advisory service. It envisions the following interaction: When the advisor wants to shift the conversation topic to the property, she uses a 3D model of a house made out of concrete and puts it in the middle of the table. She can zoom in or out and switch between map and satellite view. When moving the house token around the table, she moves the whole projection with it, such that the house token always marks the location of the property (based on its address). The map allows viewing of the facilities in the neighborhood, like schools, grocery shops, or train stations to fuel the conversation about the property. The house token & map design sources from existing practice, but extends it and moves the focus of conversation beyond numerical and technical values (size, number of rooms, year of construction, floor covering, etc.) or purely emotional topics (motivation to buy) to the location. This design augments the conversation rituals with additional data and inspires a different outlook on the property. We developed a system that supplied the house token & map (Fig. 1) functionality at the beginning of service, just after small talk.

Fig. 1.
figure 1

Map projection with the concrete house token in the middle.

Full size image

4 Methodology

4.1 Design and Implementation

LivePaper is a collaborative application for financial advisory services. It uses paper practices and augments them with projected content. The system was developed by a Swiss local bank and two universities in a joint project intended to support client-centric advisory services. It provides the means for easy generation of documentation, as well as streamlining the overall process. The whole project followed the research through design paradigm [24, 78]. Starting with practical problems, solutions were developed in a creative and iterative manner based on fieldwork, technological potentials, and models of advisory services and human interaction (e.g., theories about human introductions and greetings). Scenarios as well as clickable and functional prototypes were used to describe problems and solutions [67] and to conduct intermediate testing with key users, according to constructive design [44]. Scenario-based design and constructive design research have been widely used in Human-Computer Interaction (HCI) [77].

The house token & map design was integrated into LivePaper as one of its components. LivePaper uses a Kinect sensor to track objects on the table (sheets of paper, tokens) as well as participants’ gestures. A 4 K projector is used to project the content onto the surface of a table. Based on recognition of objects and hands, it can adjust the position of the projection, such that the content moves around together with the elements on the table. The same principle was applied to implement the house token & map: thanks to its form and a reflective sticker, the token could be recognized and its location tracked; the map appeared as soon as the token was placed in the projection area on the table between the advisor and the advisee; the map functionalities (zoom in, zoom out, switch view) were linked to buttons projected next to the map.

4.2 Evaluation

The developed prototype was tested in a realistic scenario to observe what kind of introduction rituals it affords. In particular, 24 non-IT and 24 LivePaper-supported service encounters were carried out within two weeks. They involved 8 professional bank advisors from the partner bank and 24 test advisees. The advisee participants were acquired through an announcement on a university platform (quota sampling). They were between 17 and 71 years old (avg. 32). There were 12 males and 12 females. Participants came from diverse professions, e.g., lawyer, therapist or architect. They received a short introduction to the setting and a realistic scenario to act upon. It included rough data about their financial status and data about two properties to be discussed with the advisor (one property per treatment). Each participant attended a LivePaper-supported and a conventional consultation. After both treatments, each advisee participated in an interview to report on their experiences and compare the two treatments. The experiment took place in two branches of the partner bank.

Overall, even though the observed advisory services were not real (the advisees were not real clients), by setting the context, scenarios and working with actual bank employees, we established a realistic situation for both clients and advisors. The advisors were neither aware of the property that the advisees received before the consultation, nor did they see the advisee beforehand. This preserves high ecological validity. In the conventional treatment, they used a calculator, notepad, and pen. In the LivePaper treatment, they employed the system designed as described above. Consequently, the task that participants needed to complete during the experiment was representative of and similar to what advisors and advisees do in actual advisory services at a bank, thus supporting the face validity. However, we acknowledge that external validity and transferability of the results beyond the scope of mortgage advisory services requires further research, as these experiments explicitly were designed to capture the complexity of mortgage advisory services.

The advisors used the home token & map prototype to switch between opening small talk and the core part of the encounter. Thereby they also introduced the system into an ongoing interaction: home token & map was the first feature of the system used during an advisory service. The advisors were trained in two iterations on how to handle the LivePaper: a half-day workshop one week before the test and a short training at the day of the session. The prototype training focused on use of LivePaper, but the advisor was free to decide when and how to introduce it to the interaction.

The primary data used for the analysis of system introduction sequences was collected through video recording. We recorded each advisory service with three cameras: from above, from the side and facing the advisee. Consequently, we captured the conduct on the table, bodily motions, and verbal contributions of both participants, as well as the advisee’s mimics. This allows for blow-by-blow study of the verbal and non-verbal conduct of how the participants introduce and accommodate the collaborative application. Observation of the conventional, non-IT treatment allows for comparison of rituals and performances between the two settings.

4.3 Data Analysis

The current study focuses on the initial interaction between human participants and LivePaper in mortgage advisory services. A set of qualitative methods is used: in-situ observation, coding of the video-recorded consultations, transcription of the interaction, and the interview analysis. For LivePaper treatment, the video-coding and the transcription focused on the 15 s surrounding the introduction of the system, i.e., putting the token in the center of the table. For conventional settings, the analysis focused on the introduction of a brochure as the first artefact of work. In both treatments, introduction occurs after the initial presentation of the concern by the advisee, thus yielding comparable episodes. The limit of 15 s was set arbitrarily, but considering time frames used in earlier studies on human introductions [41, 62].

These introduction episodes were first observed for all 48 consultations to create an ethogram of all performances. It considered body posture, face gaze direction, hand movements and gestures, semi-language (“oh”, “ah”), utterances and mimics. These episodes were then encoded in the video sequences. Twenty four representative excerpts were later transcribed and annotated with multimodal data according to the Jeffersonian notation [37]. The researcher conducting the coding and multimodal transcription discussed the analysis methods and results regularly with two experienced researchers.

Apart from video analysis, interviews were used as additional source of insight. A researcher coded the 24 advisee interviews focusing on statements concerning the opening of an encounter, the first use of house token & map, as well as the first use of a leaflet. The coded segments were ordered according to how positive or negative they were – the results include quotes of varying sentiments. Overall, the chosen methods capture the reactions of the participants to the introduction of the collaborative application and compare it to the introduction of traditional, material artefacts.

5 Results

5.1 Introduction Sequences

The first excerpt (Table 1) shows how a female advisor introduces the system and how a male advisee reacts. We join the participants in the seventh minute of the encounter after the advisee has explained to the advisor what property he is interested in. The advisor has finished collecting the key data in her notebook (including property’s address) and puts the writing utensils away. Thereby she clearly signals an activity shift [48]. This shift is perceptible by the client, who stops looking at the property data sheet in front of him and now observes the advisor. Advisor continues by verbally referring to “this property” and engages in several performances to introduce and use the LivePaper for the first time in this encounter. Figure in Table 1a depicts this situation.

Table 1. Introduction of the collaborative application into an ongoing encounter according to the home token & map design (15 s).
Full size table

First, the advisor searches for the house token and then stretches her arm to catch it. In parallel, the advisee starts preparing the space for something to come – he collects his documents, which had been distributed across two piles (Table 1b). However, the advisor stops her action suddenly and returns to her earlier position for a moment. She moves her notepad away as well. At that moment, the advisor and the advisee simultaneously engage in de-cluttering the interaction space (Table 1c). We observe the decluttering sequences before the house token arrives in the middle of the table. In all LivePaper excerpts the sequences were mostly initiated by the advisor, who starts decluttering her side of the table, and the advisee joins in. In Table 1, the advisee starts moving away pieces of paper directly after the advisor turns to the right and stretches her arm towards the house token. In any case, the advisor engages in behavior which signals something new to come and the advisees react by removing paper and other objects from the space between them.

As soon as the middle of the table is clear, the advisor moves the house token from the edge of the table to the middle and positions it between herself and the client (Table 1d). She uses a deictic expression and points to the house token or the map. The observed deictic expressions include “here”, “this”, “there” and “where.” Often the advisors refer to the token as “this property,” “your house.” So does the advisor in the excerpt. The advisee reacts by engaging in several behaviors in parallel: he smiles, says “wow,” gazes at the projection, moves towards the map and, as observed in this particular, case rolls up his sleeves (Table 1f). In other excerpts advisees engage in similar actions. Some advisors reorganize the interaction space even more before they proceed further with the advisee or with the system (Table 1e). The advisor alternates her gaze between the projection and the advisee. She reacts with laughter, smiling, and assuming an open posture. Having introduced the system, the advisor begins discussing the property’s location. No hesitation or long explanation phases occur – the participants continue the interaction naturally. The conversation about a property’s location may take from 30 s up to three minutes depending on the advisee’s interest. In many cases, advisees refer to the map not only verbally, but also by pointing or even touching the surface when discussing particular elements (e.g., the way to the nearest train station). After the introduction, the advisory service continues: participants discuss the options of a mortgage credit and other important features. Finally, the advisor creates one or several offerings for the client. The overall service takes up to 45 min.

The second excerpt (Table 2) illustrates how a leaflet gets introduced in a conventional setting. One observes several introduction routines here as well. The advisor uses deictic language and gestures when putting the leaflet on the table, using a single noun to describe it (“roadmap”; Table 2b). Also, she sends out signals of an upcoming activity shift (Table 2a). However, those actions are not linked to reactions from the advisee: he does not declutter the space or spend much time gazing at the introduced artifact or say “wow” or “ah.” He remains still and focused on the advisor. Even though the advisor employs some means to make the leaflet easy to read for the advisee (she flattens it, puts it in a way such that both can read it and easily point to it), the advisee’s focus remains on the advisor. He keeps his hands on the table. Overall, neither the advisor nor the advisee prepares the table or make way for the common artefact to come.

Table 2. Introduction of a leaflet into an ongoing service encounter (15 s).
Full size table

5.2 Introduction Experiences

When asked about their experiences and comparison between the two settings, the advisees often explicitly refer to the introduction of the system as a special moment. An advisee remembers the opening: “and then he places the little house and a map is shown and (…) sees everything in total – you can zoom in and out. This is an incredibly great opening. Something like this, is incredibly amazing, I was perplexed” (K18). While many clients confirm this effect, some doubt that it has impact on the pragmatic side of the advisory service: “First you see the house where it is located, just this. I mean, it is really nice that I can see it already. Just «wow»! But it is not required for the business” (K05) or “It was pretty chic at the beginning with the house to see where it is. But the practical benefit seems limited: I know where the property is I am going to buy” (K14). Still, when explicitly asked for their feelings, many advisees see the introduction as a relevant phase to generate positive emotions: “The emotions were stronger with LivePaper because we have looked at the object and where it is and how is the environment. We have zoomed out and in. During the first conventional advice, the advisor has not even asked for the street. It was really just about the purchase price and that’s it. (…) That you yourself see the object again. That you are also really sure that you really want to buy the object. This is what makes you have a positive feeling” (K15).

The advisees liked the choice to use a 3D model of a house as a token made of concrete. “You can relate to that (…) that may be a play-house, but it attracts my attention or places me in the atmosphere or environment of this house” (K25). However, this decision also generated problems. As an advisee explains: “So it was great to see where is the property is - bird’s eye view. What bothered me: it would be better if the property was not covered by the concrete house token. That was right in the middle of it. This means that one does not have a chance to look at the actual property on the map” (K05). Even though the clients notice limitations of the design, they acknowledge that it positively impacted their emotional engagement in the initial interaction.

The advisees like the fact that the introduction of the system happens with an interesting topic. An advisee puts it as follows: “The advisor was showing interest in my property (…) actually more with the LivePaper. This was so because he was really asking where it is and where it has been displayed on the map, (…) where it is exactly and what location it has, actually more with the Life Paper. The conventional service is more so serious: « I have the house and I want it and it costs so and so much » ; and then « okay, let’s start to calculate»” (K06). The observations of the frequency of advisees’ positive reactions during introduction episodes confirm the overall emotional attitude of advisees towards the interaction around the house token – see Table 3.

Table 3. The frequency of specific behaviors in the introduction sequences (15 s).
Full size table

6 Discussion

This paper’s objective was to describe the rituals involved in the introduction of a collaborative application and to compare them to other typical introduction rituals. The above analysis points to key differences concerning the introduction of collaborative material artefacts and the introduction of LivePaper, which uses the home token & map design. When introducing a leaflet, the advisor selects the correct leaflet from a folder, carefully positions it on the table, assuring that she and the client can read it, and verbally encourages the client to look at it and points at it with her hand. This sequence of actions resembles what earlier research described for a range of institutional situations [11, 16, 30, 73,74,75]. Also, the reactions of the client are not much different from what was already observed: he bends over slightly, looks at the leaflet and/or at the advisor, he shortly confirms his interest with a short “yeah.” We can confirm what literature has already stated: material supports activity shift and the advisor can use it to move the focus of the client to a specific topic, as well as to push the conversation along further [28, 31, 52, 54, 74]. Perhaps because the introduction of the leaflet as the collaborative artefact is so ordinary, no participant remarks on it explicitly in the interview.

It is quite the opposite with the house token & map. Many interviewees refer to the house token, map or their introduction even if not asked directly. In fact, advisors and advisee behave extraordinarily when they initiate the interaction with LivePaper using the house token & map design. In those introduction sequences, the advisor not only produces deictic utterances and positions the token carefully on the table, but also prepares the space necessary for it, smiles, and rearranges the space again when the projection is visible on the table. She refers to the house token as a “property” and repeats some deictic reference when presenting it (“here,” “it”). Even without comparison to the reference literature, one observes that the advisor puts forth additional effort during the introduction sequence. She behaves as if she was preparing the stage for an important element to enter the ongoing interaction. This goes beyond the usual rituals of introducing a material artefact [11, 16, 30, 73,74,75] or an artefact on a desktop computer’s monitor [13, 19, 45, 46, 57,58,59,60, 76]. In particular, it is not a straightforward turn of a monitor [2] or a positioning of material [74]. The interaction with the house token is a theatrical performance and involves emotional gestures from the advisor, such as showing both palms at the beginning of the performance or repeated pointing to the token. The client’s reactions confirm that the situation is unusual: he declutters the space, smiles, bends forward, and sometimes even dares to make physical contact with the application. Also, we observe some emotional semi-language (“oh!”, “ah!”) in addition to the more usual “yes,” “mhm”, and “okay.” Those reactions do not resemble observations reported for the introduction of material artefacts or artefacts displayed on a desktop monitor [13, 19, 60]. Something essential is different.

We argue there is a shallow and a deeper explanation for the reported results. The shallow interpretation relies on comparison between the typical human introduction as analyzed in ethnomethodological studies [14, 41, 52, 56, 61,62,63,64]. They share a lot: participants adapt new, open positions; they are directly referring to the newcomer by explaining meaning and background; there is an intensive gazing towards the newcomer; and, finally, there are welcoming gestures, touches, and emotional semi-language expressions. It seems that the home token & map design affords rituals otherwise employed when introducing and greeting humans. But what makes the simple design elements (a concrete house model and a map) more social than the leaflet?

The house token & map design, in fact, implements some features that make the system and its elements easier to embrace by the participants than the leaflet (or any generic element like a piece of paper or a desktop monitor):

  • The house token is meaningful based on its form and has a clear relation to the content of the ongoing interaction. It simply stands for the property, which the advisee is going to buy. The advisee can easily decipher the meaning of the object when the advisor moves it to the center of the table. It is similar to what happens when a human newcomer enters an ongoing interaction: the pre-present participants perceive the newcomer’s intention to join based on the context and her or his apparent role. Before the newcomer enters, the pre-present participants plot a story in their minds about what the newcomer’s intentions and expected behaviors are. Think of a situation when a beggar approaches someone on the bus stop with their hand stretched out – one can easily predict what to expect. This is not possible when a monitor gets turned or when a brochure enters the stage during an advisory session: those elements do not provide enough data for the advisee to craft a meaningful plot.

  • The house token and the map afford deictic, physical references. It seems natural to the participants to point at the house as a physical object and to denote locations with “here” and “there.” It resembles what happens when a newcomer joins an ongoing interaction, and someone introduces them: pointing toward someone (or something) in our physical space belongs to the standard repertoire of communication. While pointing at objects displayed on the screen or in a brochure is possible as well, it requires more precise pointing, because these things are smaller and often more numerous (screens and brochures contain multiple images). Using a pen to point to a small element seems more natural then, while pointing with a pen to a person or larger object is unusual. Due to hand-sized elements House token & map promotes physical reference with gestures involving hands, fingers or a pen.

  • Finally, the design affords movement in space to signalize activity shift. And this movement is very specific: the advisor moves the house token into the center of the table from the edge. This horizontal trajectory resembles the situation when a newcomer approaches a pre-existing group of people. When someone turns a monitor around or pulls a piece of paper from a pile or a folder, the object moves along a different path and this movement is shorter. In fact, starting the house token nearer the center was not possible because it would directly turn on the map and disturb the participants’ small talk. Admittedly, this technical limitation contributed to the theatrical character of the introduction sequence, but it does not explain such behaviors as decluttering the space or showing open palms. Overall, the advisor’s movements marked activity shifts and signalized a new element coming to the stage.

According to this shallow interpretation of the results, the above design decisions make participants attach a social character to the introduction of the house token & map. This would align with the “computers-are-social-actors” paradigm: the participants conduct actions otherwise typical for ritual introduction of a human actor. They prepare a dedicated space, produce emotional semi-language or attempt physical contact in a careful and respectful manner. It seems as if the participants were following typical schemata for introducing a (small) human individual. One could claim that the results even extend the previous paradigm. Originally, the CASA paradigm claimed that users transfer human attributes to technology because modern technologies use language as input and output, are interactive, or replace typically human roles [66]. The current results suggest that due to a hunch or intuition that an interactive technology might come to the table, users apply introduction schemata reserved for humans. This is surprising, given the fact that previous research on introducing computers has not reported similar observations [1, 13, 57]. There are two possible explanations for this difference: (1) previous studies have not topicalized the social character of the observed behavior or (2) humans treat computers like social actors only in specific circumstances. Identifying and describing those circumstances would require further research. This study suggests that the physical form and emotional value of the computer may play a role in this regard.

However, consideration of what a social ritual is and why it emerges provides a complementary, yet deeper explanation. Ritualized performances in interaction with other humans emerge not only through mindless repetition. Humans replicate rituals because, through those behaviors, they can show respect to the other party (or at least pretend to do so) [27, 40, 70]. This understanding helps interpret the results. For instance, an advisee’s decluttering is a ritual response to advisor’s efforts: when the advisee notices the advisor reaching for something or decluttering her side of the table, he is driven to acknowledge this by making space for what is coming. The advisee’s smiles and emotional utterances show not only interest and pleasure, but also signalize acknowledgement of the advisor’s efforts. Finally, advisee’s bending forward or pointing towards the map do not follow only from curiosity in the content, but acknowledge the advisor’s effort. Following this line of argumentation, the advisor’s actions can be seen as a consequence of the physical affordances of the house token & map design and the reactions from the advisee as ritualized acknowledgment of the advisor’s work. The collected data does not provide a conclusive answer as to which explanation is the right one. Of course, these two mechanisms (computers-are-social-actors and ritualized respect signs) are not incompatible; the observed behaviors can be driven by both of them.

7 Conclusion and Limitations

The presented insights do not come without limitations. First, we rely on data collected in a realistic, but experimental context. While this helps us to compare the encounters and identify patterns in predefined conditions, it also generates questions about external validity. Replicating this study with clients in real advisory services outside the mortgage context could produce more insight, especially because of potentially stronger emotional engagement. Second, the results rely on field observations and video material. We report on observed regularities rather than making quantitative claims about them. Measuring the occurrences of particular performances might be the next step to further support the findings. Third, the transferability of the identified design rationales still needs to be confirmed. Analyzing whether doctor-patient or teacher-student encounters exhibit similar patterns requires design and observational research.

All in all, this paper provides a genuine, focused analysis of how participants introduce and “welcome” a collaborative application. The paper makes several contributions: It identifies atypical behaviors in introduction sequences for a collaborative application thus extending previous literature on initial interaction with computers [1, 13, 57]. It relates the observations to the implemented design decisions, thereby adding to the research on supporting advisory services [17, 42], which did not explicitly consider initial interaction or MR technology so far. Finally, it offers interpretation of the rituals in light of two social interaction theories and provides supporting evidence from realistic, controlled setting rather than from abstract experiments or anecdotes [55, 66]. It also compares those behaviors to the introduction sequences derived and consolidated from ethnomethodological research. The study highlights a crucial but neglected aspect of technology use – the “welcoming” phase, provides new perspectives on it, and offers both theoretical and practical interpretations of the observed behaviors.