Abstract
With the emergence of multi-screen services, this new technological advancement allowed users to consume their desired content according to their needs and through different contexts. Prior research from the field have shown that users were able to identify specific personalities for products. However, personality was determined mostly by product appearance and the single device approach was not adequate in explaining the experiences with multiple devices. Therefore, this study aimed to explore three multi-screen interaction experiences, tackle users’ emotional responses and investigate what kind of interaction personality they perceived during interaction. The result of this study demonstrated the relationship between user perceptions and multi-screen experiences, which is useful for future design reference in multi-screen services.
Keywords
1 Introduction
Report has shown that consumer owns an average of four digital devices [1], which means that the number of touchpoints has expanded with the rise of device ownership. Multi-screen services like Netflix, Allrecipes and Fitbit have broken usage boundaries and allowed users to freely shift from one device to another. Multi-screen services involve consumer interacting with multiple devices and is characterized by the different location, duration and purpose of use compared to single screen services. Multi-screen services have formed new kinds of interaction experiences: consistent, continuous and complementary experiences [2]. Thus, the user-device relationship now involves more than one device and has become even more complex. According to CASA (Computers are social actors) paradigm, users tend to interact with computers as if they are human beings and apply social rules (e.g. politeness) to them [3]. This study aims to take it further by investigating the relationship between users and multi-screen interaction experiences, with emphasis on users’ emotional responses to the experiences and personality traits they assign to the interactions.
2 Literature Review
2.1 Interaction Experience
When devices operate within a multi-screen service, the relationship on how devices connect with each other to accommodate different services or to conform to the users’ behavior, have formed new kinds of interaction experiences. How people interact with multi-screen is different from the traditional way of interaction with single device. Levin has revealed three kinds of interaction experiences: consistent, continuous and complementary experiences [2]. Consistent interaction refers to the same experience interacting with similar contents, visual effects and main functions with different kinds of devices. These characteristics decrease people’s learning curve when migrating through devices, and people can intuitively and easily adopt the best available device to be used according to their wishes. Continuous interaction emphasizes on its continuation in accomplishing a single or sequential of activities across various devices; therefore, the coordination between devices is determined through users’ activity flow, duration of the activity, context of the activity and the number of subtasks needed to be accomplished. Activities like cooking, trip planning or reading books on multiple devices can be done in a more seamless and sophisticated way of interaction. Complementary interaction is the most representative interaction of the multi-screen ecosystem. Two or more devices in an activity work together, or complement with each other or control one another to elevate the overall experience. It is the most accessible for users since users can use their own device as a starting point and extend the experience beyond what was already in existence. For instance, using one’s own smartphones as a way of input in a multi-player game so everyone who owns a smartphone has the opportunity to participate in the game and thus promote social connections. Also, devices could serve as second-screen to provide additional information to enrich the main activity such as receiving movie description and viewers’ comment on a tablet that corresponds to the movie that is currently being watched [4].
These three kinds of multi-screen interactions have provided various impacts, and the experiences can only be amplified and accommodated when users’ behaviors have been identified and when the actual needs were fulfilled. Prior research have emphasized on multi-screen user behavior such as energy intake [5] or human activity for system recommendation [6]. There is a deficiency in the understanding of the interplay between multi-screen interaction and user perceptions. Höök et al. stated that the overall experience of interaction could not be explained fully by the knowing of only traditional human factors; however, users’ perceptions such as emotion, understanding, and interpretation are also as equally as important [7]. Therefore, this study aims to identify the emotions and personality traits perceived from multi-screen interaction, which has never been done in this aspect before and to provide more information from the user perception point of view.
2.2 Emotion in Multi-screen Interaction
Emotions play an important role in human-machine interaction as stated by Norman [8], and this has been extensively studied and analyzed in HCI (Human-computer interaction). From a user experience point of view, emotional responses can be influenced and determined by interpreting the iterative interplay between the context and the continuous actions [9]. The emotions identified and analyzed in this study are the results and consequences of interactions with multiple devices, which is different from the concepts of seeing emotion as an antecedent or mediator to product use [10]. Most often, emotional responses are associated with the needs and goals of an individual. In an information-seeking context, if the user interfaces that users interacted with can help to navigate or facilitate in finding desired information, users’ emotions are often positive. That is, users’ emotional experience can be directly impacted by the user interfaces to the extent of conforming to their needs and goals [11]. Therefore during the context of multi-screen services, interaction experiences occurred with the desire to accomplish certain goals, so users’ emotional responses would be determined by whether the interaction could successfully assist users toward those goals. Although it is hard to say what kind of emotions users would experience from these multi-screen experiences, but certainly they would influence users’ emotions [12]. For this reason, this study assumed that different emotions would be elicited from the different interactions in consistent, continuous and complementary experiences. Identifying the emotions from interactions in multi-screen services can help to determine which interaction to assign to a given activity that is consistent with users’ emotional expectations or the demand of the services. The study of the non-instrumental aspects through emotions and personality traits enriches the current field of research with more knowledge on how people perceive this kind of newly emerged interaction experiences.
2.3 Personality in Multi-screen Interaction
Based on different contexts of use in multi-screen services, people tend to personify their devices with certain personalities and roles that go beyond their instrumental power. For example, tablets are interpreted as an on-the-move professional assistant in formal business context [13], and devices are described as companions [14, 15]. Personality traits applied to products are originally used in describing human beings, and users are able to differentiate them among products [16]. Since people interact with devices similarly to how they interact with human beings, being aware of their device personalities can help them better mange tasks successfully, just like how people in social interactions with each other [17]. Most often, a person’s personality can be assessed from his or her facial expressions, personal styles or postures, and a product’s personality can be perceived from its shape [18] or material choices [19]. Since appearance is an important determinant in the perception of personality in products, user interfaces and computers, it is unclear whether the same theory applies to multi-screen interaction where more than one device are involved. Desmet et al. have analyzed the relationship between interaction, appearance and personality in physical interactions, which demonstrated that appearance is strong in interaction personality, but personality is not merely the sum of product appearance and the effect of interaction [20]. Therefore, it would be interesting to probe into the attributes constituting personality in multi-screen interactions. This study aims to understand people’s perception toward multi-screen interaction, comprehend whether the interactions can be described with personality traits, and to explore the relationship between user perception and the three kinds of interaction experiences.
3 Method
In order to get users’ real feedback, the experiment simulated the real world setting, and invited participants to physically interact with the devices. The purpose was to find out users’ perception towards consistent, continuous and complementary interaction experiences through post-interaction questionnaire and a follow-up interview to answer whether personality traits can be discovered in multi-screen interactions and the relationship between user perception and three kinds of multi-screen interaction in respect of emotion and personality.
3.1 Participants
A sample of 16 participants (9 male and 7 female, mean age = 28.63) participated in this experiment. Majority of them had experiences in using two to three devices along or in conjunction with one another in completing a task. On a daily basis, participants owned and used smartphones (100 %), tablets (44 %), laptops (81 %), desktops (81 %), Internet-enabled TV (13 %), smart-watches (6.3 %) and other wearable devices (6.3 %).
3.2 Questionnaire
A paper-based post-interaction questionnaire was used to collect participants’ impressions after completing each task. The questionnaire includes questions regarding emotion and personality descriptions. For personality assessment, a set of 24-item personality traits was adopted from previous studies [20–23]. Participants’ emotional responses were assessed with the positive and negative affect schedule (PANAS) [24]. Questionnaire data were analyzed using Ward’s hierarchical clustering method, and selected by means of the cluster and variation. Therefore, ten items in total were retained to be used in the experiment: cheerful, open, relaxed, easy-going, cute, interesting, dominant, boring, gentle and preference for personality scale; and strong, enthusiastic, inspired, determined, attentive, active, interested, distressed, hostile and irritable for emotional scale.
3.3 Apparatus
The hardware used in this experiment includes an Apple iPhone 4 (iOS version 9.2.1), a 42-inch TV, a 13-inch MacBook Pro and a 13-inch Asus laptop. The user interface images shown in the experiment were designed previously in graphic design software. Since appearance is a major determinant in product personality [18], in this experiment the appearance of user interfaces were controlled, so the design elements in all images remained identical in terms of color, icon and style. The images were then imported to and simulated on smartphones with POP prototyping application (Woomoo Inc.), which enables the simulation of applications participants would interact on their smartphone. For the simulation on laptops and TV, Microsoft PowerPoint was used to simulate webpage browsing and TV watching. One of the essential features of multi-screen interaction is that it is context sensitive. Therefore, the experimental setting divides the room into three distinct areas: living-room area, study room area and an outdoor area (Fig. 1). Participants were informed about the purpose of each area before the experiment took place.
3.4 Task Procedure
The scenario of the experiment was developed based on the topic: exercise. With the emerging number of people using multiple devices like smartphone, laptop or wearable devices (e.g. Nike fuelband, Fitbit, Apple watch, etc.) to record, keep track and share their physical activities, exercise is a good example to demonstrate the multi-screen service therefore it was chosen for this study. Based on the scenario, three tasks were designed according to three multi-screen interactions of consistent, continuous and complementary design. In order to simulate the interaction and usage between two to three devices together, the Wizard of Oz (WOz) technique was applied. During the experiment, participants were told that the devices they are using were all connected together in the multi-screen ecosystem by a service called Myworkout, and in the following three tasks they were going to interact with different kind of interaction provided by the service (Table 1).
As the experiment was a within-subjects design, participants were required to perform all three tasks. The experiment was carried out in an individual basis. Participants were assigned to three different task sequences (e.g., ABC, BCA, CAB). The entire experiment lasted approximately 30–40 min.
4 Result
The method by Ward’s hierarchical clustering was used to analyze the post-interaction questionnaire. The cluster analysis was later ascertained by one-way ANOVA to see if there was significance among the clusters and to determine the level of the clusters.
4.1 Personality Clustering
In consistent interaction, the items regarding personality are grouped into four clusters: 1st cluster (cute, boring), 2nd cluster (interesting), 3rd cluster (cheerful, easy-going, gentle) and 4th cluster (relaxed, preference, open, dominant). In continuous interaction, the items regarding personality are grouped into four clusters: 1st cluster (cheerful, relaxed, easy-going, preference), 2nd cluster (open, dominant), 3rd cluster (cute, gentle) and 4th cluster (boring). In complementary interaction, the items regarding personality are divided into four clusters: 1st cluster (open, easy-going, relaxed, cheerful), 2nd cluster (dominant, interest, preference), 3rd cluster (cute, gentle) and 4th cluster (boring).
4.2 Personality in Multi-screen Interaction
In consistent interaction, the clusters differed significantly in terms of personality (F3,156 = 31.914, P = 0.000 < 0.001). With the LSD post-hoc test applied, it was revealed that cluster 4 is larger than the other three clusters. Participants thought consistent experience had a relaxed, preference, open and dominant personality. In continuous interaction, the clusters differed significantly in terms of personality (F3,156 = 40.195, P = 0.000 < 0.001). With the LSD post-hoc test applied, it was revealed that cluster 2 is larger than the other three clusters. Participants thought continuous experience had an open and dominant personality. In complementary interaction, clusters differed significantly in terms of personality (F3,156 = 37.72, P = 0.000 < 0.001). With the LSD post-hoc test applied, it was revealed that cluster 1 is larger than the other three clusters. Participants thought complementary experience had an open, easy-going, relaxed and cheerful personality (Fig. 2).
4.3 Emotion Clustering
In consistent interaction, the items regarding emotion are grouped into four clusters: 1st cluster (determined, active, attentive, interest), 2nd cluster (enthusiastic, inspired, strong), 3rd cluster (distressed, irritable) and 4th cluster (hostile). In continuous interaction, the items regarding emotion are grouped into four clusters: 1st cluster (determined, attentive, enthusiastic, inspired), 2nd cluster (active, interest), 3rd cluster (strong, distressed) and 4th cluster (irritable, hostile). In complementary interaction, the items regarding emotion are grouped into three clusters: 1st cluster (inspired, interest, active, enthusiastic, determined), 2nd cluster (strong, attentive) and 3rd cluster (irritable, hostile, distressed).
4.4 Emotion in Multi-screen Interaction
In consistent interaction, the clusters differed significantly in terms of emotion (F3,156 = 53.122, P = 0.000 < 0.001). With the LSD post-hoc test applied, it was revealed that cluster 1 is larger than the other three clusters. Participants felt determined, active, attentive and interested. In continuous interaction, the clusters differed significantly in terms of emotion (F3,156 = 103.52, P = 0.000 < 0.001). With the LSD post-hoc test applied, it was revealed that cluster 2 is larger than the other three clusters. Participants felt active and interested. In complementary interaction, the clusters differed significantly in terms of emotion (F2,157 = 127.38, P = 0.000 < 0.001). With the LSD post-hoc test applied, it was revealed that cluster 1 is larger than the other two clusters. Participants felt inspired, interested, active, enthusiastic and determined (Fig. 3).
5 Discussion
The purpose of this study is to understand user perception in multi-screen interaction experiences, in particular what emotions would be perceived as well as the personality traits that users would assign to the interaction. From the result of the experiment, similarities were discovered among three interaction experiences: consistent, continuous and complementary. Participants thought all multi-screen interactions possessed an open personality that is perceived to be curious, creative, flexible and more opportunities. An open personality is part of the openness trait in the five-factor model described by Carver and Connor-Smith [25]. They demonstrated that openness trait is related to engagement coping, which refers to problem solving and cognitive restructuring. That is, the three multi-screen interactions have the tendency to provide positive impacts as the interactions serve to assist participants in accomplishing their goals and accommodate stressor in the activity process. In terms of emotion for the three kinds of interactions, similarities were also found as participants thought all multi-screen interactions elicited emotions that are active and interested. It was found that all of the interactions have been mentioned to be impressive, and participants felt interested and attracted toward the interaction they thought to be the most impressive and wanted their devices to interact in the same way. Overall, the multi-screen interactions are perceived to be open, active and interested.
The results of the experiment showed that different multi-screen interactions tend to associate with certain personality traits and emotions (Table 2). In consistent experiences, preference and attentiveness are the most representative personality and emotion in this experience. Preference usually occurred when the design elements are in concordance with the characteristics of its user group [22]. It can also be explained from the interview conducted, as participants’ prefer consistent design in icon position, layout and style that facilitate easier searching and positioning across multiple devices. The experience was perceived to be deliberate and provide users with a sense of familiarity. Therefore, consistent experience is suitable to apply to services dealing with enormous information content or complex information architecture when users are expecting simple, familiar interaction experience with minimum adjustment required across multiple devices.
Result showed that continuous experience is perceived to be open, dominant, active and interested. The term dominant is used to describe a person exhibiting tendencies to command, control or lead others to take action, and a dominant interaction is usually perceived from the order of interaction, the confidence level and the language displayed on the computer [26]. In the services where continuous experience takes place, users often got asked to make certain decisions for the actions to be performed across multiple devices for continuation of the tasks. A too dominant interaction may make users feel imposing and forceful, while a less dominate interaction may be too weak and disorganized. It is therefore vital for designers to take into consideration the personality traits of the experience and their impact since these may interfere with users’ reaction to an activity. Services dealing with user behaviors and contextual information such as scheduling or quantified self movement, the spontaneous and convenient characteristics of continuous experience may be preferable to enhance such services. The term “convenience” has been emphasized several times by participants to address the freedom of the interactions. The technical foundation of continuous experience thus needs to be well designed so that every device can work smoothly with on another to provide the freedom of device shifting for the users.
In the complementary scenarios, users perceived this kind of experience as cheerful, easy-going, enthusiastic and inspiring, which were perceptions not perceived in either consistent or continuous interaction experiences. In an information-seeking approach, an easy-going person usually prefers easy access to information that requires less effort and thoroughness [27]. Therefore, complementary interaction experiment is perceived with a spontaneous, uncomplicated and relaxed feeling. Users could utilize their existing devices to collaborate or control other devices. This low entry barrier reinforce the anytime, anywhere concept, which makes users feel emotionally positive and inspires them to pursue further activities with other devices. Moreover, the combination use of multiple devices not only deepened the content knowledge in a service, but also elevated users’ experiences to another level that are novel, attractive, smart and well-executed. With these perceived characteristics, complementary interaction experience is best adopted to enhance the fun experiences in entertaining services, and to enrich with constant inspiration and motivation in exercising or productivity services.
6 Conclusion
In multi-screen services, interaction experiences were the results derived from the contexts of activities, the number of available devices, and users’ personal goals. The results of this study showed that user perception could be elicited from multi-screen experiences, and variations of emotions and personalities can be found among consistent, continuous and complementary interactions. As mentioned earlier in this study, users interact with computers through social norms similarly to human interactions. As a consequence, future research could investigate relationships among user interfaces, interaction emotions and personality traits in the multi-screen services. This would verify the effects on whether the attributes of user interface such as color, input/output, message route, or information content are compatible with the types of interaction experiences.
The characteristics of the different interaction experiences may serve as a criterion to be adopted in an activity, as it is undeniable that certain interaction is more suitable for certain activity. This study wants to aggregate on top of what has already known and provides additional insights from user perspective for designers to adopt the appropriate interaction to match with their design goals. It is also useful to enhance existing services so that the conveyed interaction emotion and personality is in consistence with the purposes of the services. Furthermore, it can be utilized as an evaluation tool to examine the discrepancies between user perception and the services they use. Even though the subject of this study focused on the issue of exercise, the experiment was conducted through simulating real world situations under well-controlled experimental conditions. Therefore, results from this study can be further analyzed and used in different subject matters, providing further insight in the field of multi-screen interactions.
References
Nielsen: A Nielsen report on the digital consumer (2014). http://goo.gl/Af7vI2
Levin, M.: Designing Multi-Device Experiences: An Ecosystem Approach to User Experiences Across Devices. O’Reilly Media, Sebastopol (2014)
Nass, C.I., Stener, J.S., Tanber, E.: Computers are social actors. In: Proceedings of CHI 1994, pp. 72–78 (1994)
Cesar, P., Knoche, H., Bulterman, D.C.A.: From one to many boxes: mobile devices as primary and secondary screens. In: Marcus, A., Roibás, A.C., Sala, R. (eds.) Mobile TV: Customizing Content and Experience. Human-Computer Interaction Series, pp. 327–348. Springer, London (2010)
Marsh, S., Mhurchu, C.N., Jiang, Y., Maddison, R.: Modern screen-use behaviors: the effects of single- and multi-screen use on energy intake. J. Adolesc. Health 56(5), 543–549 (2015)
Schweizer, I., Schmidt, B.: Kraken.me-Multi-device user tracking suite. Presented at the 2014 ACM International Joint Conference, New York, USA, pp. 853–862 (2014)
Höök, K., Persson, P., Sjölinder, M.: Measuring experience of interactive characters. In: Green, W.S., Jordan, P.W. (eds.) Pleasure with Products: Beyond Usability. Taylor and Francis, London (2002)
Norman, D.: Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books, New York (2004)
Battarbee, K.: Co-experience: understanding user experiences in social interaction. Unpublished doctoral dissertation, University of Art and Design Helsinki, Helsinki, Finland (2004)
Hassenzahl, M., Tractinsky, N.: User experience - a research agenda. Behav. Inf. Technol. 25(2), 91–97 (2006)
Brave, S., Nass, C.: Emotion in human–computer interaction. In: Jacko, J., Sears, A. (eds.) The Human-Computer Interaction Handbook, pp. 81–93. Lawrence Erlbaum Associates, Mahwah (2003)
Chorianopoulos, K., Spinellis, D.: User interface evaluation of interactive TV: a media studies perspective. Univ. Access Inf. Soc. 5(2), 209–218 (2006)
Zamani, E., Giaglis, G.M., Nancy, P.: Dad bought another toy: meaning making and emotions with tablets. In: 2012 International Conference on Mobile Business, Paper 16 (2012)
Turkle, S. (ed.): Evocative Objects: Things We Think with. MIT Press, Cambridge (2007)
Van Dijck, J.: Mediated Memories in the Digital Age. Stanford University Press, Stanford (2007)
Jordan, P.W.: The personalities of products. In: Green, W.S., Jordan, P.W. (eds.) Pleasure with Products: Beyond Usability. Taylor and Francis, London (2002)
Janlert, L.E., Stolterman, E.: The character of things. Des. Stud. 18, 297–314 (1997)
Govers, P., Hekkert, P., Schoormans, J.P.L.: Happy, cute and tough: can designers create a product personality that consumers understand? In: McDonagh, D., Hekkert, P., Van Erp, J., Gyi, D. (eds.) Design and Emotion, The Design of Everyday Things, pp. 345–349. Taylor and Francis, London (2004)
Van Kesteren, I.E.H., Stappers, P.J., Kandachar, P.V.: Representing product personality in relation to materials in a product design problem. In: 1st Nordic Design Research Conference, Copenhagen, Denmark (2005)
Desmet, P.M.A., Ortíz Nicolás, J.C., Schoormans, J.P.: Product personality in physical interaction. Des. Stud. 29(5), 458–477 (2008)
Govers, P.C.M.: Product personality. TU Delft, Delft University of Technology (2004)
Fogg, B.J.: Persuasive Technology: Using Computers to Change What We Think and Do. Morgan Kaufmann, San Francisco (2002)
Schenkman, B.N., Jönsson, F.U.: Aesthetics and preferences of Web pages. Behav. Inf. Technol. 19, 367–377 (2000)
Watson, D., Clark, L.A., Tellegen, A.: Development and validation of brief measures of positive and negative affect: the PANAS scales. J. Pers. Soc. Psychol. 54(6), 1063–1070 (1988)
Carver, C.S., Connor-Smith, J.: Personality and coping. Annu. Rev. Psychol. 61, 679–704 (2010)
Moon, Y., Nass, C.: How, “Real” are computer personalities? Psychological responses to personality types in human-computer interaction. Commun. Res. 23(6), 651–674 (1996)
Heinström, J.: Five personality dimensions and their influence on information behaviour. Inf. Res. 9(1), 165 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Liao, IC., Deng, YS., You, HC. (2016). The Emotion and Personality User Perception in Multi-screen Interaction. In: Marcus, A. (eds) Design, User Experience, and Usability: Novel User Experiences . DUXU 2016. Lecture Notes in Computer Science(), vol 9747. Springer, Cham. https://doi.org/10.1007/978-3-319-40355-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-40355-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-40354-0
Online ISBN: 978-3-319-40355-7
eBook Packages: Computer ScienceComputer Science (R0)