1 Introduction

The perceived audience of a social network post consists only out of 27% of the actual audience [1]. This leads to a high amount of unwanted recipients of the posts information, which can be used for various attacks like stalking, identity theft, user manipulation (also known as “social engineering”), re-identification in other anonymized data sets or face re-identification [8]. The success rate for a stranger to be accepted as a facebook friend and thus be added to the post audience is surprisingly high: In a study from Gross and Acquisti, 75.000 out of 250.000 users accepted the friend invitation from an unknown person [8]. Social networks such as Facebook or Google+ allow their users to create custom friend lists and share content exclusively with these list. However, these tools are often not used [20]. Known causes for this behavior are the mental effort to group people [29] and usability problems, e.g. regarding the mechanics, general workflow or simply user-interface-related problems [15, 16, 30].

Compared to other sorting tasks, the task of social network friend grouping includes several special challenges to be solved: First, privacy preferences have shown to be highly individual in several domains like location sharing [22], mobile app permissions [24] or shopping scenarios [25] as well as the task of friend grouping [23]. Every user has her own different criteria to build groups and to categorize her friends into them, depending on her personal preferences, her personality and privacy attitude [23], and also her posting preferences, regarding post topics and intimacy of the shared information [23]. Second, there is no definite answer on the correct assignment of a friend. Some of the friends might fit into multiple groups; some might not fit in any group and will remain unassigned. This leads to the fact that for some friends, it is directly clear to which groups they should be assigned to, whereas the user needs a longer time to think about a correct assignment for other cases, as our study results will show.

Research tried to tackle this problem by creating new design concepts with an increased usability in order to reduce the mental effort to perform the friend sorting task. Some of the approaches use graph-based interfaces [7, 19], where groups are represented by vertices with the corresponding friends attached as their leaves; others rely on a conventional list-based design [18] improved by an auto-grouping algorithm based on community detection [2]. Nevertheless, the usage of virtual reality to enhance the usability of social network friend sorting on the one hand, and making the task more interesting and enjoyable by enhancing the user experience on the other hand, has not been discussed in research so far to the best of our knowledge. To be more precise, we try to solve the following research questions:

  1. 1.

    Can we enhance the usability and user experience of the social network friend sorting using a VR environment and metaphors?

  2. 2.

    Do users prefer a playful or a pragmatic approach for VR sorting?

  3. 3.

    How do the VR designs effect the errors made during friend sorting?

For this purpose, we created two different UI designs, one focused on further increasing the usability in a virtual reality environment by adapting traditional concepts such as card sorting (“pragmatic design”), and one that is geared towards making the sorting task as fun and enjoyable as possible by packaging the task as an interactive VR game (“playful condition”). In a study comparing these to a conventional sorting interface from the Facebook social network website, we found out that we could further increase the usability with the pragmatic design. The playful condition was perceived as highly motivating and achieved a significantly higher user experience score, at the cost of an increased error rate.

2 Related Work

For the scope of our work, there are three research fields of interest that we want to discuss in this section: first, the usage of social network friend lists, strategies used and problems that arise within the current solution; second, user interface designs in other domains that enhance usability; and third, the questionnaires that are of importance for this research field and the later evaluation study.

2.1 Friend Grouping in Social Networks

A comparative user study by Kelley et al. [16] in 2011 investigated different metaphors and user strategies that arise when users start grouping friends into friend lists on the Facebook social network. Their user study included three new interfaces that have been compared to the Facebook UI: a card sorting method using printed pictures of the friends; a grid tagging approach where all friend images are printed in a grid shape on one page, so that the user can use pens of different colors representing the friend groups the friend should be assigned to; and a file explorer where the friends are represented as files that are sorted into folders using the Windows explorer. Lastly, they used the current Facebook interface as a reference interface. In a user study, they found out that there are two user strategies: the “by friend” strategy, where the groups needed for this friend are created first, and then populated by all other friends that fit inside these groups before the user proceeds to the next friend; and the “by group” strategy where all needed groups are created first, and then populated by the friends one after another. As some of their design recommendations, they advised to always keep in mind that Facebook friend grouping is not a primary task, and secondly, that the grouping changes over time. In our work, we took up these design recommendations and created a user interface that motivates the user and that packages the uninteresting sorting task as a challenging VR game, so that the task is done more frequently, especially over time.

Other studies about usage frequencies and mental effort have shown that the usability and understanding the grouping interface is not a problem for social networks like Google+ [14, 29]. However, the mental effort required for finding a good assignment is very high, leading to the fact that users prefer to censor their posts rather than to use friend grouping to select the correct audience [14, 29]. According to a study by Javed et al. including 200 participants, more than 50% of all Facebook users have not created any personal friend lists at all. Another 15% and 10% have created only one or two friend lists, respectively [12]. Interestingly, automatic friend groups have a small amount of overlap between them, whereas user-created friend groups have little to none: Out of all members of a friend list, 90% are not present in any other friend list for user-created friend lists compared to about 58% for automatically created lists. Usually, users never add a friend to more than two friend lists; some are not added to any friend list. On average, each self-created friend list contains about 32 members. Based on the mentioned study results, we designed our UI designs so that being able to add friends to multiple lists should be easily possible, but is not a primary design goal of the UI.

2.2 Sorting in Virtual Reality and Other Domains

In general, the sorting task or categorization task described here involves several items that have to be assigned to one or multiple groups by the user, based on the individual features of each item. There are several abstraction levels of the items to be sorted that can be used for a sorting task, each with its own advantages and drawbacks [26]. The most concrete form is object sorting, where the user is given concrete objects to be sorted. Although this provides the most information for the user, it can contain irrelevant and distracting features as well, that may lead to a different evaluation and assignment of the object than intended. The next abstraction level is picture sorting, where the object is represented solely by an image of the object. Although this reduces the amount of sensory input, it is possible to trim out unimportant features from the picture, reducing the amount of irrelevant information for the user. Lastly, card sorting uses only written text on a card describing the object. While this design is most restrictive and allows limiting the information only to the sorting features that are of relevance, it can only be used for items that are known to the user, and that he can imagine from reading the description.

Apart from the level of detail, sorting techniques also differ in the group creation policy used for sorting items into groups using a desktop interface. Card sorting approaches [21] can be differentiated between “closed card sorting”, where the categories are already pre-defined, and “open card sorting”, where the categories are defined by the user during the sorting task. Whereas the former leads to more comparable results between subjects, the latter gives the user more degrees of freedom, which makes it more suitable for tasks with individual data, like the task of friend grouping. Finally, the UI designs for card sorting also differ in the design metaphor used to display and arrange the cards. There are several commercial applications that use a stacked card sorting metaphor, where the cards of the same category are arranged on top of each other. Examples are CardZortFootnote 1 or OpenSortFootnote 2. Other designs like WebSortFootnote 3 use a more explorer-like approach, where the categories are displayed as a vertical list of terms that can be opened like folders in the Windows explorer. Card sorting tools have been proven to be highly efficient, but the preferred design greatly depends on the user group [5]: although researchers perferred the explorer-like approach in WebSort, most of the end-users liked the stack-based approach of OpenSort best. Studies have shown that there are no significant differences in terms of performance when comparing online and paper-based card sorting [4]. Sorting is also possible in VR, as a study on “how similar looking products influence the overall performance in a retail setting” has shown recently [11]: The participants were shown a VR world, where they had to sort products arriving on a conveyor belt into two different categories. Although the participants were generally succesful in the tasks, the error rate significantly increased when the similarity of the products was increased.

For the task of social network friend sorting, the abstraction level of “card sorting” might not be sufficient, as users might not remember the actual person behind a username, since users often use fake names for their online profiles. Apart from that, we cannot rule out that two or more social network friends might have similar images, leading to an increased error rate. We therefore decided on an improved “card sorting+” design, which includes a profile picture and a textual description using the username as well. The task of social network friend sorting is highly individual, which led us to the decision to use open card sorting, using predefined groups for convenience with the possibility to add additional groups as needed. As most of the end users opted for the “stacked card metaphor” for arranging the cards, we based our interface designs on this metaphor as well.

2.3 Measuring User Experience, Usability and Other Aspects Important for Our Work

The de-facto standard for measuring usability is the system usability scale (SUS) [3], which was introduced by John Brooke in 1986 to provide a way of “quick and dirty” quantification of a perceived usability. The SUS generates a score between 0 and 100, where a score above 68 is perceived as “good” usability. Although often used for measuring usability, the questionnaire captures only the pragmatic quality, e.g. the usability of an interface, without considering the hedonic aspects, e.g. the user experience regarding the experienced stimulation and fun when using the UI. The AttrakDiff questionnaire [10] contains questions capturing the pragmatic aspects, as well as the hedonic aspects like identity and stimulation of a user experience, and is geared especially towards comparing different interfaces to each other with regard to usability and user experience. The AttrakDiff generates three different scores that are of interest for our design experiment: The pragmatic quality (PQ) capturing the usability, similar to the SUS and two hedonic scores describing the stimulation perceived when using the interface (HQ-S) and how much subjects could identify with the UI (HQ-I). All scores range from −2.5 to 2.5, where a value above 1.0 or 2.0 is perceived as “good” or “excellent”, respectively. According to the goals of our experiment capturing usability as well as user experience in a comparative study, we decided for the AttrakDiff questionnaire in our case.

Capturing the perceived workload including factors like mental or physical demand and frustration is mostly done using the NASA TLX questionnaire [9]. The questionnaire has been developed over a three-year development cycle with more than 40 lab experiments and is now cited by more than 4400 studies, denoting the widespread influence of this questionnaire [6]. The questionnaire measures the mental demand (e.g. cognitive load), physical demand, temporal demand (e.g. perceived time pressure), the perceived effort needed to achieve the desired goal, as well as the perceived frustration and the subjects’ own achieved performance estimated by the subject. Each single value ranges from 0 to 100 where 0 is the lowest (best) and 100 the highest (worst) workload. We included the questionnaire in our study especially to compare the mental and physical demand between the standard interface and the VR designs, and the frustration experienced with the different interfaces. Lastly, we added the motion sickness assesment questionnaire (MSAQ) to our experiment to ensure that motion sickness does not affect our UI designs in a negative way. Scores on this measure range from 11.1 for no motion sickness to 100 for highest motion sickness effects.

To conclude, research has explored several different metaphors for sorting, whereas the metaphor of card sorting has been proven to be highly effective. Related work has shown that VR creates a higher immersion compared to traditional interfaces, resulting in a feeling of being “in the game” [13, 27]. VR applications can, if done well, lead to the most natural and most efficient interaction, far better than it can be achieved with traditional 2D or 3D applications [13, 27]. Research has also proven that using VR for e-commerce applications like VR shopping is perceived as significantly more useful (in terms of interaction techniques), immersive and interesting by users that its two-dimensional counterparts [17, 28]. These results leads us to the assumption that VR can also improve the usability and perceived fun of a friend grouping task, which we are eager to test in this paper. In our work, we build upon the idea of card sorting and transfer this metaphor into a VR design to enhance usability on the one hand, and the user experience on the other hand, to present the task of social network friend grouping as an interesting and enjoyable task that is still perceived to be easy to carry out with our designs.

3 FriendGroupVR Designs

We implemented two different VR design approaches to sort and organize social network friend lists, targeting different objectives: The first “pragmatic” approach is optimized towards usability in terms of efficiency and performance, whereas the second “playful” approach is focused on maing the sorting task as enjoyable and interesting as possible. Each world was implemented in Unity using an HTC Vive VR Kit. The setup contained a 4 m \(\times \) 4 m floor equipped with an HTC Vive Lighthouse setup that allows tracking the user’s movements inside the area (see Fig. 1). Each user movement was reflected in the VR world as well. In order to track hand movements, each user was given two Vive controllers, one for each hand. Grabbing gestures were realized by usage of the trigger buttons of the controllers. For our lab study, we recorded the created friend lists and the contained friend lists locally instead of applying the changes to the user’s social network account.

A special problem of the friend sorting task is that the time needed for the assignment of a friend to one or multiple groups can be highly variable. For some of the friends, it is directly clear to which social circle(s) or which friend list(s) they belong, but for others it is less clear, so that the user might need some seconds to think before he can conduct the actual assignment task. Therefore, we put a special emphasis on the possibility to interrupt the sorting task between two friends, so that the user has the possibility to think about the best assignment options in advance.

Fig. 1.
figure 1

VR setup in our lab using the HTC Vive. “L” denotes the positions of the lighthouse position trackers, “V” the initial position of the user wearing the HTC Vive

Full size image

3.1 Pragmatic Design

According to related work, the metaphor of card sorting is one of the most efficient methodologies [5], we therefore decided to transfer the open card sorting metaphor into a VR world, leading us to an office metaphor as shown in Fig. 2. The “cards”, i.e. the social network friends, are represented as picture frames (“friend frames”) standing inside a bookshelf. Each friend frame consists of the friend’s profile picture and forename on the front, and the fore- and surname on the back, forming a combination of a card sorting and picture sorting metaphor (“card sorting+”), as described in the related work section. Friends can be displayed with ascending tie strength (equivalent to the Facebook friend list order) or sorted by fore- or surname. As space is limited, the shelf always contains only nine friend frames at a time. To access the other frames, we placed two buttons at the left and right edge of the shelf, allowing the user to access friends that appear earlier or later in the sorted list, respectively.

Fig. 2.
figure 2

Bookshelf in the pragmatic design, including friends represented by “friend frames” (left) and spawner to create new friend list boxes (right).

Full size image

According to the stacked cards metaphor, friend lists are represented as labeled boxes (“list box”) in which the user can drag & drop friend frames using a VR controller. As a starting point, the VR world contains the five most frequently used friend lists according to related work [23], namely “family”, “acquaintances”, “close friends”, “work” and “sport”, as a box. Boxes have no physical weight in our VR world, and can therefore be placed in mid-air at any desired location. As the task of arranging friend lists is highly individual, we opted for an “open card sorting” design allowing users to create arbritrary additional friend lists. To create a new list box, we added the “box spawner” into the environment (Fig. 2): To create a list box, the user has to touch the red button with the VR controller, which opens a VR keyboard to enter the list name. Pressing the enter button hides the keyboard and spawns the newly created list box, as seen in the figure.

To manage the friend lists, a user typically starts with creating and arranging the list boxes around the shelf. After that, the friend frames are traversed one after another and placed inside one or multiple list boxes that should contain the friend. As soon as a friend frame is placed inside a box, the frame is shrunk to half of its size to save space. If a friend is placed inside the wrong box or if the user decides to assign them to a different list box, he can always empty the box on the floor or grab a picture inside the box and put it into another.

3.2 Playful Design

In contrast to the former design, we concentrated on making the sorting task as joyful and interesting as possible. We therefore decided to design the approach as an interactive VR game that challenges the user, including gamification elements like high score tables, upgrades and bonus items that should motivate the user in conducting the task and competing with others. As stated in the beginning of the section, the time needed for finding an optimal assignment is very different from friend to friend. We therefore need a game design which can be interrupted or delayed at certain points in time to allow the user to take her time for the assignment decision. As related work has shown, most of the friends (about 90%) are assigned only to one friend group; we therefore decided on a game with a linear action line, where only one friend is part of the game at a time, with the possibility to manually go back to a friend again if he or she has to be added to multiple friend groups. We came up with the idea of a “can knockdown” game, where the user can assign her friend to friend lists by shooting dispatched “friend balls” to different can stacks representing the available friend lists. Using this design, the user can always wait and think about the correct assignment, before she starts the dispatch of the friend ball.

In a typical workflow, the user first creates the needed friend lists using a tool similar to the box spawner in the pragmatic design. After this task is finished, the friend lists are represented by can stacks (“list stacks”) at a distance of about five meters in front of the user. The user then starts the assignment phase, where a ball representing each social network friend is dispatched in the direction of the user one after another. The user uses a bat to redirect the friend ball to a can stack corresponding to the friend list the user has to be assigned to. As mentioned before, each friend ball is dispatched only once. If the user wants to add a friend to multiple groups, he has to press the “back” button on the control panel (see below) to display the last friend ball again and add her to another group. Depending on how many cans the user is able to hit with the friend ball, the user gains points to be added to his personal high score.

Fig. 3.
figure 3

Can knockdown game in the playful design.

Full size image

A screenshot of the playful VR world from the user’s initial position can be found in Fig. 3: The shelf on the left side of the user (Fig. 4 left) is used to create and arrange the friend lists, similar to the pragmatic design. In the shelf, friend lists are represented by a small board with the list name written on the front. Similar to the other design, the five most frequently used friend lists are already created in advance and placed at the bottom of the shelf. If the user wants to create a new friend list, he touches the button, which opens a keyboard to enter the friend list name, exactly like in the pragmatic design. To use a friend list in the can knockdown game, the user has to place a friend list board in one of containers in the shelf, which will display a can stack in the game at the respective location (e.g. if the board is placed at the container to the left of center, the corresponding can stack will also be shown to the left of center in the game). At the right hand side of the user is a control panel (Fig. 4 top right) which allows the user to switch forward or backward between the social network friends, and a button to start and pause the game at any given time, for example if more time is needed to contemplate the correct friend list assignment. Using the control panel, the user can also go back to an earlier friend and dispatch her friend ball another time to assign him to another friend list.

Fig. 4.
figure 4

Friend list management shelf (left), score and dispatch panel opposite the user, dispatching a friend ball (bottom right) and control panel to switch current friend and start/pause the game (top right) in the playful design. (Color figure online)

Full size image

To the front, the user is facing a panel (Fig. 4 bottom right) which displays the name and profile picture of the next friend to be sorted, together with the current score and the remaining time for the currently collected bonus item (see below). When the game is started by pressing the “start button”, the user has five seconds to think about the correct list stack that he wants to aim at. The five dots at the top of the panel represent the time in seconds that is remaining. When the last dot turns from grey to red, the friend ball is dispatched towards the user. If no can stack is hit, the same friend ball is again dispatched for another try. When the user does not want to assign the friend to any group, he can aim for the monster at the upper left of the VR world, which will then eat the friend ball, so that it is not dispatched again. If the user hits the wrong can stack, he can always undo the last assignment by hitting the “undo” buzzer directly in front of him. To further motivate the user, we integrated “bonus balls” into the game, which are dispatched in the direction of the user at randomized times. Collecting each bonus ball activates a special upgrade for a limited time, for example a score multiplier, or an increase of the friend ball size.

Fig. 5.
figure 5

Different bats available to the user with ascending difficulty from left to right.

Full size image

The user has a choice of different bats (Fig. 5) with different difficulties: The easiest bat is largest and catches the ball so that the user has the possibility to aim and shoot at the desired location by pressing a button. The second easiest bat has the same size, but directly reflects the ball without catching it first. The remaining two bats have the same behavior with a smaller size, making it more difficult to hit the ball. The more difficult a bat is, the more score is rewarded for each can hit. When all friend balls have been processed, the game stops and the user’s high score is displayed on the high score table in the upper right of the VR world, along with the high scores of other users, and the friend lists are stored.

4 User Study

We had the goal to find interaction designs using VR for the creation and maintenance of social network friend lists, that would be both more efficient and also more enjoyable than the current standard. In order to measure the differences from the Facebook UI, we conducted a lab study at our department, where the participants had to use both VR designs as well as a standard interface from the Facebook social network site using a desktop PC as a baseline. With each interface, the participants had to assign their 40 closest friends (according to the Facebook friend ordering) to friend groups. For each condition, we recorded usability and user experience scores using the AttrakDiff [10] questionnaire as well as an error rate (for example friends missing from a group, or friends assigned to the wrong group), as described below in more detail. To reduce training effects and to get users used to the VR environment, we implemented another vr “training” world which shows the user an overview of the 40 friends that have to be assigned in the experiment (Fig. 6). To further reduce training effects, the order of conditions was permuted for each participant so that each sequence of conditions appears equally often during the study, leading to \(3!=6\) different orders.

Fig. 6.
figure 6

Training world with 40 friends that the user is shown before the experiment starts.

Full size image

The procedure was the same for each participant but with a different order of conditions, as stated before. After signing a consent form and the privacy policy, the participant had to fill in a questionnaire about demographic data and previous experience using the Facebook friend grouping tool and virtual reality setups. She was then given a desktop screen to enter her Facebook login data. With the aid of the Selenium web browser automation toolkitFootnote 4, a Python script then traversed the participant’s friend list and extracted the friend names and profile pictures for later use during the study. After the process was finished, the participant was given instruction in the VR hardware, and had to put on the headset for the first time. We started the training level and gave her the time to get familiar with the VR world and the controllers, and to have a first look at the friends to be sorted and to contemplate the friend lists and the assignments to be made. When the participant stated she was ready, the training world was closed, and the main experiment phase started.

In the main phase, the three interface conditions were tested one after another in a different order, as stated above. For each condition, the participant was given an introduction to the interface with all of its interaction possibilities and some time to get familiar with it and to test each functionality once. When she stated she was ready, the world was reset, and the participant had to do the friend grouping with her 40 friends until she stated she was finished. Participants were told that they should do the task seriously, as wrong assignments would be recorded. In the following, the participant had to fill in several questionnaires about the current condition: the AttrakDiff questionnaire [10] measuring usability (PQ) and user experience (HQ-I, HQ-S), the NASA TLX capturing the mental and physical workload, an MSAQ questionnaire asking about motion sickness in the VR conditions as well as a custom questionnaire asking whether the interface was motivating or fun to use, and whether the participant thought it could be integrated into her daily life, on a five-point Likert scale. After a five-minute break to rest and recover, this procedure was repeated for the two other conditions. For each condition, we recorded the overall time spent on sorting. At the end of the study, the participant was asked which was their favorite interface, and had to traverse the friend lists created to check for errors made during the assignment. We recorded the following error measures:

  • Person missing from a group (MISS)

  • Person added despite not belonging to the group (TOOMUCH)

  • Wrong group label (LABEL)

  • Group should be split into multiple groups (SPLIT)

  • Multiple groups should be merged to one group (MERGE)

5 Results

In total, we had 30 participants in the study, 18 female and 12 male. Participants were recruited at our university using postings and the university’s social network group. As a compensation, a €25 Amazon voucher was raffled off among all participants. The age ranged from 19 to 50 (mean = 26.67, SD = 5.474), representing a good portion of typical social network usersFootnote 5. When asked about their experiences with virtual reality, 12 people had no experience (40%) and 5 almost no experience (16.7%). 25 people answered that they had never used Facebook’s grouping interface (83.3%), while 5 people had used it (16.7%). On average, the main experiment was completed within 64 min.

The experiment results can be found in Table 1. Depending on whether an F-test showed a normal distribution of the data, we performed pairwise paired T-tests or Wilcoxon signed-rank tests to compare the pragmatic quality (PQ), also known as usability; the measures from the custom questionnaire asking about experienced fun (FUN) and motivation (MOTIV) and suitability for daily use (DAILY), the NASA-TLX workload values, times needed for sorting, the error rates and the hedonic scores HQ-I and HQ-S measuring the user experience between the three conditions.

Table 1. Results for the usability and user experience scores including pragmatic quality (PQ), hedonic quality regarding stimulation (HQ-S) and identification (HQ-I) and the custom questions asking about fun (FUN) and motivation (MOTIV) to do the task and suitability for everyday usage (DAILY), as well as the Nasa TLX workload, time spent on sorting and the error rate.
Full size table

The usability (PQ) was highest for the pragmatic interface (\(M=1.99\)) and significantly better than for the playful interface (\(M=0.61, T=4.6, p<0.001\)) which is itself significantly more usable than the Facebook standard (\(M=-0.15, T=3.1, p=0.004\)). Regarding the user experience, the user could identify significantly better with the pragmatic interface (\(M=1.20\)) than with the playful interface (\(M=0.0.96, T=2.13, p=0.042\)) which was again better than Facebook (\(M=-0.53, T=8.38, p<0.001\)), but felt most stimulated by the playful interface (\(M=2.04\)) followed by the pragmatic VR design (\(M=1.13, Z=-4.50, p<0.001\)) and distantly followed by the Facebook UI with a significantly lower score (\(M=-1.81, T=14.471, p<0.001\)). The FUN was on average also highest using the playful design (\(M=4.77\)) although we could not prove the difference to be significant from the pragmatic interface (\(M=4.57, Z=0.965, p=0.334\)). The Facebook interface was again rated significantly worse than the pragmatic interface (\(M=1.57, T=4.79, p<0.001\)). The most motivating interface is the pragmatic interface (\(M=4.33\)) according to the mean values, but is again not significantly better than the playful interface (\(M=4.23, Z=1.62, p=0.09\)). The Facebook UI is again significantly worse than the playful UI (\(M=1.65, Z=4.75, p<0.001\)). The same order holds for the suitability to integrate the UI into everyday social network usage: The pragmatic interface (\(M=3.77\)) significantly outperforms the playful interface (\(M=2.87, Z=3.24, p=0.001\)) which is again significantly better than the current standard on Facebook (\(M=2.10, Z=2.39, p=0.017\)). The time in seconds needed to perform the grouping task was lowest with the Facebook interface (\(M=418\)) and significantly higher with the pragmatic (\(M=587, T=2.722, p=0.011\)) and playful VR designs (\(M=588, T=2.50, p=0.018\)). A visual analysis on the time distributions for the times needed to assign a single friend showed that the times are very different for some of the users, supporting our assumption that an interface is needed that allows users to pause the sorting task, as the time needed for an assignment can differ. Figure 7 shows the time distribution for three representative subjects of the study.

Fig. 7.
figure 7

Distribution of the time needed to assign a friend for three representative subjects.

Full size image

The motion sickness (MSAQ) scores (ranging from 11.1 = best to 100 = worst) were very low for both VR interfaces and did not differ significantly (\(M_{pragmatic}=16.02, M_{playful}=16.37, Z=0.991, P=0.322\)), attesting that motion sickness was not a noticeable problem in our UI designs. The pragmatic interface received on average the smallest error rate (\(M=7.60\)). Nevertheless, the error rate using the baseline interface is not significantly higher (\(M8.50, Z=0.419, p=0.675\)). The playful interface led to the highest error rates, which are significantly higher than for the standard Facebook interface (\(M=11.933, Z=2.204, p=0.027\)). The same holds for the workload, which is highest for the playful interface (\(M=39.42\)) and significantly lower for the Facebook UI (\(M=30.61, T=2.79, p=0.0009\)) and lowest for the pragmatic UI (\(M=22.85, T=2.90, p=0.007\)). A detailed overview on the error rates and the different workload items can be found in Tables 2 and 3. We can clearly see that the main cause for the higher workload in the playful design is that the VR game was perceived as challenging, as the mental demand (\(Z=4.19, p<0.001\)), temporal demand (\(T=4.37, p<0.001\)) as well as the effort (\(Z=4.22, p<0.001\)) is significantly higher compared to the pragmatic interface. The frustration was highest using the Facebook interface, supporting our claim that friend grouping is perceived as a very frustrating and uninteresting task. Using VR, the frustration is significantly smaller for both the pragmatic (\(T=6.79, p<0.001\)) as well as the playful design (\(T=3.65, p=0.001\)). The most favored interface was the pragmatic design (73.3%) followed by the playful design (23.33%). Only one participant claimed to like the standard Facebook interface best. We observed different behaviors regarding the choice of the used bat throughout the game: 17 participants used the “sticky” bat and 9 used the “reflective” most of the time (\({>}80\%\) of the time), three switched between both. One used the baseball bat exclusively. However, we did not find any significant difference between these usage groups for any of our measures.

Table 2. Detailed results for the average number of errors per participants.
Full size table
Table 3. Detailed results of the NASA TLX.
Full size table

6 Discussion

6.1 Increased Usability Using VR

We presented two VR friend grouping interfaces, one geared towards maximizing the usability, and one towards maximizing the fun and motivation when sorting the friends. Comparing the usability scores of the interfaces, we can see that both VR interfaces were perceived as significantly more useful than the Facebook interface. The pragmatic design achieved the highest usability score of 1.99 which is very close to the theoretical maximum of 2.50 on a scale from −2.5 to 2.5, indicating that we achieved the goal of increasing the usability compared to the current standard. Interestingly, the interaction time was lowest for the Facebook interface, although it was rated with a significantly lower usability. On the other hand, the error rate was higher for the Facebook interface, leading to the assumption that the Facebook interface is fast to use on one hand, but is complicated and leads to an increased error rate on the other hand, which leads to a smaller perceived usability of this interface.

6.2 Challenging Game Design Leads to Increased Error Rates

The error rate is on average lowest for the pragmatic interface, although the difference to the standard interface is not significant. As stated earlier, we designed the playful design to be challenging for the user, including bonuses, high scores, and different levels of difficulty using different bats. This is also reflected in the perceived workload according to the NASA-TLX, where especially the mental and physical effort is higher compared to the other interfaces. Nevertheless, the frustration is low compared to the Facebook interface, indicating that the stress was perceived to be positive. However, the challenges may also lead to the increased error rate, which is also highest for the playful interface. The game may have been too challenging, or the design as a game may have led the participants to take the task less seriously and pay less attention to a correct sorting; which of these factors led to the increased error rate should be further investigated in a follow-up study.

6.3 Significantly Improved User Experience, Not only for the Playful Condition

The differences for the user experience scores are again larger than for the usability scores when comparing the VR designs with the standard interface. The playful design received a very high stimulus and FUN score, again indicating that the game was perceived as challenging, stimulating and fun to use. But the pragmatic design, which was not optimized towards user experience, also achieved a high user experience score, which was significantly higher than for the Facebook baseline. The pragmatic design was voted to be most motivating, although not significantly more so than the playful interface. One reason why it was rated to be more motivating on average might be the good combination of an appealing and interesting user interface while still providing a high usability without trying to challenge the user. Which factors led to the higher motivation should therefore be investigated in the follow-up study.

6.4 Conclusion on the Optimal Design and Favored Interface

Taking all the aspects into account, the results indicate that VR designs are perceived as more useful on the one hand, and as more fun and motivating on the other, which gives them a clear advantage over the current mouse & keyboard interface. However, such conventional interfaces have the advantage that every computer is equipped with a mouse and keyboard; the audience that can use the Facebook interface is therefore currently significantly larger than those who own a VR kit at home to do the friend sorting with one of the two VR designs. Nevertheless, VR interfaces will gain importance in the next few years, as the number of VR users is increasing exponentiallyFootnote 6. As stated in the introduction, one of the major problems of friend sorting is that the task requires a high mental demand, making it a task that is often avoided. A first approach is therefore to shape the task as an interesting and challenging game, like our playful design. However, as the results show, a playful design always has the drawback that it can lead to the task being taken less seriously or being lost in the game without paying attention to the actual task, leading to a decreased quality of the outcome of the task. Therefore, according to our results, it seems that the optimal way, and the way preferred by users, is a VR design which is targeted towards usability and that could be enhanced with some small game elements, but without losing the focus on the actual task too much. These results confirm the study findings about card sorting, which was already shown to be very efficient using a desktop interface [5], and which seems to be efficient for sorting within a VR world as well. Regarding the differences in time needed for assigning a friend to a list, our results indicate that this time indeed is very diverse, making it important to design a user interface or a VR sorting game so that it can be paused at any time, especially between the items to be sorted. Whether these assumptions can be proven to be true remains for a follow-up experiment, where we will take a closer look at the effects that led to the increased error rates in the playful condition.

6.5 Limitations and Future Work

The aim of this paper was to have a first look at how virtual reality interfaces can help in making the task of friend sorting more fun and interesting, leading to possibly higher usage rates of friend lists in social networks. We created two different designs, one trying to improve the usability of the sorting using VR metaphors, and one geared towards making it a more fun and entertaining experience. Based on different criteria like the possibility to interrupt the sorting task at any given time or by using gamification elements to enhance the user experience, we came up with two different designs for our user study. Notwithstanding, plenty of other possible design ideas exist and might be suitable for this kind of task. Nevertheless, we were able to show that, with our design ideas, the user experience as well as the perceived usability could be improved. However, we would like to elaborate on other designs in the future, especially game designs that might be more prone to errors than our can knockdown game, although the increased error rate might be an effect of the gamified design, which would hold for other game types as well.

In the experiment, we used the current standard interface as a baseline to minimize side effects and to get a comparison of our designs to the current working standard. Although we were able to prove that both usability and user experience were higher using the VR design, we would like to elaborate more on the parts of the design that lead to this effect. We would especially like to discover which of the developed metaphors led to an increased rating; whether it was the representation of the friends as friend frames inside a shelf, the friend boxes or the interaction by inserting the frames inside the box. Also, the alone usage of virtual reality might already lead to some effect. In several follow-up studies we would like to find out more about which design elements have a positive effect in VR using A/B testing, and give concrete guidelines on which metaphors should be used, and which should be avoided. Finally, we would like to integrate our work into a social network, so that VR users can try the solution in their everyday social network usage. We are especially interested in acceptance and usage rates: whether the app is accepted after the first usage, or whether they fall back to using the Facebook interface after some time, and whether the usage of a VR app increases the frequency of social network friend sorting.

7 Conclusion

Neglecting privacy settings in online social networks can lead to serious harms, but privacy functionalities like friend lists are rarely used in social networks, because the mental effort for creating friend lists prior to their usage is too high, leading users to either censor their posts or to publish more information than they intended to. Related work focused on improving the usability of conventional desktop interfaces for friend sorting. In our paper, we took a first look at how friend sorting interfaces could look in virtual reality. We proposed an interface focused on usability by taking the idea of card sorting into vr, and a second interface having the goal to maximize the user experience by wrapping the sorting task in a challenging game. A comparative study with the Facebook sorting interface as a baseline has shown that both interfaces achieved their goal of improving the usability and user experience, although the error rate significantly increased within the playful design. However, which distinct factors led to the increased error rate, and which factors led to the increased user experience scores, should be further studied in future research.