Keywords

1 Introduction

With the transformation of our surroundings in a technology-enhanced environment, driven by ambient data-processing and data-sharing, humans face continuous privacy decision-making. One strategy to reach teenagers/young adults who are more likely to share fake information [1] and make less reflected privacy choices [2], is creating awareness by video games. Learning by playing games as well as learning through creating games are well explored and are often applying co-creational activities involving the card and board metaphor [3]. Research has shown that co-located workshops utilizing paper cards facilitate discussion, support generating new ideas/knowledge [4] and help optimizing concepts [5]. They have also proven to be valuable in co-design scenarios for eliciting tacit knowledge and integrating the perspectives of multiple disciplines/perspectives [6]. These qualities are of particular interest for the design of Serious Games for educational purposes.

As pointed out by Dörner et al. [7], Serious Games are games with the intention to entertain and to achieve at least one or more additional goals. In the case of privacy awareness, for example, factors such as risk-taking behavior [8] or peer pressure [9] may be assessed in the game to improve both the game experience and the efficacy of the learning game. Thus, complementing the educational goal with a researching goal. At the same time, disruption through in-game assessment or extraneous cognitive influences from interaction design should be avoided in such analytic educational games to preserve an engaging game flow [10]. While card-based tools have proven helpful in co-located/classroom education and co-creational design activities, their benefits in the increasingly important distance learning scenarios are under-explored. This study thematizes the application and benefits of the card and board metaphor in the case of co-creating games for privacy education via videoconferencing.

1.1 Card-Based Co-creation of Educational Game Concepts

Considering the educational, analytical, and interactional aspects in a Serious Game (SG) from an early stage of game design can help maintain an engaging game experience. Prior work has shown that participatory approaches to the co-creation of SG concepts can support ideation of game challenges and help to balance SG concepts.

Fig. 1.
figure 1

Paper-based co-creation toolset for ideation of analytic educational games

Full size image

A card-based toolset – the Challenge Game Frame (CGF) – that integrated the perspectives of teachers/researchers with the roles of the player and interaction designer through affordance analysis has been demonstrating the applicability and balance/ideation support in a previous study [5]. The paper-based co-creation toolset (Fig. 1) consists of game design suggestion cards for the player, teacher, researcher and interaction designer roles and an associated board to lay out a balanced Serious Game concept. It can be applied in classroom settings with students and teachers for co-creation of analytic learning games.

The collaborative design approach empowers all involved roles to discuss conflicts and adapt the game challenge to generate a harmonious game concept. The current educational game design toolset provides card decks for each role – 12 decks in total. The role-oriented decks (affordance cards) for the player consist of achieving, acting, progressing, engaging, and adapting while the teacher decks comprise reflecting and examining, the researcher card decks cover researching, reporting, and monitoring and the interaction designer decks include interacting and presenting suggestions.

To set the frame for optimizing the game concept, the toolset additionally features 4 role-independent card decks with design suggestions to define the game’s context, target group, and educational domain. The design proposals in these decks outline who will play the game, where and when the game is planned to be played and what challenging privacy issue is addressed as the domain goal. Privacy challenges described in detail include, for example, unnoticed third-party data sharing, the risk of aggregation/profiling, and large-scale tracking of health or private behavior. Each of the toolset’s 150 cards describes a suggestion on how to actualize/design a part in the educational game concept. With the provided design suggestion cards, playing board and stepwise playbook/instructions, non-game experts such as student groups are supported in collaboratively ideating and balancing the game concept for engaging privacy education [5].

However, a non-digital toolset is constrained to offline/co-located use cases while educational institutions are increasingly faced with technology-enhanced remote collaboration scenarios. This paper addresses the digitalization of the card-based offline activity to learn more about remote co-creation with the card and board metaphor. Applicability and ideation/balancing support are studied by comparative co-located and remote co-creation sessions with student groups co-creating educational privacy game concepts.

1.2 Related Work and Research Gap

Non-digital card toolsets for collaborative idea generation have been studied intensively over the last decades, as summarized by Wölfel and Merritt [11] and more recently by Roy and Warren [12] as well as Aarts et al. [13] and Peters et al. [14]. Several card toolsets address gamification [15] or general game design [16]. Similar to other tools, the paper-based CGF supports students in on-site classroom settings in the co-creation process but additionally includes the learning and assessment perspectives. As with other ideation sets [4, 17], using the card metaphor and a scaffolding board structure supported discussing, ideating and balancing the concept [5]. However, rapidly evolving online learning situations and current social interaction restrictions originating from the covid-19 pandemic prevent much of the utility of physical, co-located ideation. While many educational institutions shift to online learning and remote seminars, physical collaboration toolsets cannot be applied. Yet, there is a paucity of research that addresses remote co-creation with student groups. Only few researchers addressed remote ideation or synchronous online co-creation scenarios. One study looked at paper sketching via Skype [18] and found that face-to-face communication remains important in remote collaboration. Stockleben et al. [19] point out in this regard that traditional emotional cues are diminished in remote collaboration and suggest implementing substitutes in the online tool. Ho and Tomitsch [20] recently investigated brainstorming tools for collaborative game ideation and found most strategies at game jams rely on paper or whiteboard. They concluded that there is a need for idea sharing and arrangement platforms with strategies to overcome issues of unguided idea creation sessions. In their integrative model created from a recent review of electronic brainstorming, Maaravi et al. [21] emphasize the benefits of clear goal descriptions for ideation. Their suggestions include assigning quantity and quality goals and setting success criteria. Further, they recommend combining asynchronous and synchronous ideation phases, encouraging working individually, having explicit discussion guidance as well as rank-order procedures and mechanisms to maintain motivation. However, the focus of their work was on settings not using face-to-face communication and idea generation via text messages/typing.

Despite the substantial research on traditional collaborative card ideation, no studies were found to investigate the transfer of a paper-based card-toolset to a digital representation for remote co-creation and the corresponding effects on user support and applicability.

1.3 Research Objectives

This present study addresses the outlined research gap and investigates the effects of remote co-creation with digitized cards and playboard. By comparing the co-located paper-based card toolset and the digitized remote co-creation variant, implications concerning support for ideation, balancing and applicability are researched. Additionally, the co-created concepts are analyzed for frequency patterns to learn more about the potential impact of the distinct toolsets and students’ preferred approaches to privacy games. The research questions guiding the investigation were accordingly:

  1. 1.

    How does remote co-creation with digital cards and playboard support ideation, concept balancing, and toolset applicability compared to a paper-based co-located variant?

  2. 2.

    What are the preferred privacy education game patterns co-created by students using the remote and co-located toolsets?

2 Research Approach

In our work, we followed the cyclic design science research model [22]. The described paper-based card and board toolset (Sect. 1.1) is thereby transformed in this design cycle into a digital representation/artefact. The card decks, the role tokens and the playboard instructions are digitized to an online tool that supports remote co-creation (Sect. 2.1) of educational game concepts. The digital card-toolset is planned to be applicable in combination with video chat tools used in distant education settings. In the subsequent evaluation/relevance cycle, the technology-enhanced version and the non-digital toolset are evaluated in a between-subject trial. Both variants are applied by student groups for co-creating engaging privacy education games (Sect. 2.2). The user trials were evaluated by pre/post questionnaires assessing perceived support for ideation, balancing and application of the two toolsets. According to the first research objective, the null hypothesis established for the empirical investigation was:

H0::

‘There are no significant differences in perceived support for game concept ideation, balancing or applicability between co-located and remote card-based co-creation.’

To address the second research objective, the outcomes of the classroom and remote co-creation sessions were comparatively analyzed to determine design choice frequency patterns for each card deck.

2.1 Digitizing the Paper-Based Card Toolset

The following paragraphs describe the transformation of the paper-based toolset to the technology-enhanced digital tool. The digital cards, board, and instruction process were designed with the aim to closely retain the card and board metaphor for the online context.

Technical Background.

As the digital toolset is planned to be used together with video call software and run on all platforms without restrictions, it was chosen to be a web browser application. However, it should also be possible for future mobile adaption to build it as an app for all popular mobile platforms. Moreover, flexibility regarding the visual representations of cards, boards, and roles was an aim since improvements must be easy to implement during the iterative design science cycles. Therefore, the technical base was chosen to be the Unity Engine (unity.com). Data storage was realized by secure communication via https to a MySQL database under the researchers’ authority.

Cards.

The card and card deck metaphors were kept in the transformation process in a general approach of skeuominimalistic aesthetics [23]. Thereby, the proportional shape, fonts and colored frame of the cards were kept. On the other hand, there was no attempt to simulate paper card depth with shadows (Fig. 2).

Fig. 2.
figure 2

Selecting paper-based cards and digital card selection

Full size image

While in the paper-based toolset, blank cards were provided for writing own suggestions, the digital version featured a “+” button to add own ideas to a deck. The technology-enhanced ideation tool is thought to be used in distant learning and with video conferences on the desktop/notebook. As not every device in this context possesses a touch screen enabling swipe gestures, the card browsing featured previous/next buttons.

Board.

Similarly, the digital playboard was closely oriented on the design of the paper board. The game concept was presented as one stream from left to right with slots for each role-oriented card suggestion (Fig. 3). For each card deck, one slot was reserved in this main game challenge stream (stream A).

Fig. 3.
figure 3

Placing cards on the paper-based playboard and on the digital playboard slots

Full size image

To encourage card matching and discussion, slots for an alternative stream B were presented below stream A. Placement of cards in the slots was translated to the digital board as drag-and-drop action. To ensure that all co-creators could follow the events, card movements were synchronized over the network.

Roles and Playbook.

The role tokens of the paper-based toolset were digitized as displayed in Fig. 4. As emotional cues between participants are diminished in remote settings [19], a mechanism for expressing content/happiness with the current concept was integrated. Each group member/role was provided with the possibility to state their happiness with the current concept using a rating slider (0 to 100%). The picture of the digital role avatar changes accordingly from a neutral to a happier expression. The combined group satisfaction (mean) for both alternative game concepts (A/B) was also displayed synchronously for all co-creators. As Maaravi et al. [21] proposed, this may serve the remotely collaborating group as a quality success criterion.

Fig. 4.
figure 4

Role token in the physical toolset and digitized role token including feedback sliders for the co-created concept

Full size image

Clear goal description and stepwise instructions are recommended by several researchers for non-digital [4, 17] and digital collaborative idea finding [21]. The paper-based toolset features instructions and time for each step on the bottom of the A0 playboard (Fig. 5, left).

Fig. 5.
figure 5

Stepwise instructions in the paper-based toolset and stepwise digital instructions

Full size image

Utilizing the benefits of a digital representation, time for the step was implemented as a countdown (Fig. 5, right). To synchronize guidance for a group working remotely, the instructions were announced by a digital instructor character before each step (Fig. 5, center).

2.2 Co-located and Remote Co-creation of Privacy Education Game Concepts

Both toolsets were applied in trials with student groups. Two classes of Computer Science students (n = 61) at a Norwegian university participated in the co-creation sessions. In one class (n = 29) the paper-based toolset was applied by eight co-located groups in the classroom. The seven groups in the other class (n = 32) were holding physically distanced remote sessions via video conference and using the digital co-creation toolset (Fig. 6).

Participants.

Before the activity, participants (aged 20 to 29) provided informed consent and indicated their game design and software development skills via questionnaire on a seven-point Likert scale (1, none; 7, professional). Both classes showed a comparable skill average in game design (remote M = 2.7, co-located M = 2.2) and software development (remote M = 4.3, co-located M = 4.1).

Fig. 6.
figure 6

Co-located session with the paper-based toolset (left) and remote co-creation using the digital card toolset (right)

Full size image

Procedure and Data Collection.

The task for the student groups was to co-create an engaging game that educates about privacy issues with players working collaboratively together in the game. The co-creation sessions were organized for two hours in total. Participants were first introduced to Serious Game Design theory and the balancing process between game goal/domain goal and engagement/assessment. Secondly, the basic principles of the toolset were explained. After this 30-min introduction, participants filled out the pre-questionnaire to report their skills and proceeded to the one-hour co-creation activity. Each step of the co-creation process was to be completed in a certain number of minutes. For the paper-based toolset, these minutes were written on the playboard, and the lecturer announced when to proceed to the next step. The remote ideation tool stopped being interactive when the step time expired, and a defined group manager could initiate the next step for all participants by clicking a button on the playboard. Each step time and step instruction were inscribed on the bottom of the paper-based playboard. For the online toolset, the step instructions were displayed at the beginning of a step and read out by a voiceover before the group manager clicked the start button. The steps in both, remote and co-located co-creation were as follows:

  1. 1.

    Pick a role of either player, teacher, researcher or designer

  2. 2.

    Define the context of the game: domain, target group, location/time of play

  3. 3.

    Individually read through role-assigned cards and pick favorites

  4. 4.

    Co-create/balance a game challenge: starting from left to right, discuss ideas from cards or create custom cards

  5. 5.

    Identify conflicting pairs in the game concept and balance out the potential flow breaks by discussing alternative picks or another group agreement

  6. 6.

    Agree on the final picks, define a working title and write a game plot summary

Finally, after the co-creation session, all participants filled out the post-questionnaire individually on their experience regarding tools, activity, and outcome. A seven-point Likert scale (1, strongly disagree; 7, strongly agree) was used to assess the participants’ judgement. The questionnaire items asked about perceived ideation/balancing support and perceived applicability of the combined toolset, cards, roles and playboard. While the ideation and application dimensions consisted of four items each, balancing support was assessed with eight items. Exemplary items/statements included: ‘Using the cards helped to focus on ideas’ (ideation); ‘The roles helped identifying conflicts between the game parts.’ (balancing); ‘I can imagine using the game design tool on my own for group co-creation’ (application).

3 Results

Data analysis showed that collected data was not consistently normally distributed but displayed comparable distribution and homogeneity of variance. As suggested by Field [24], non-parametric Mann-Whitney U analysis (α = 0.05) was performed and showed significant differences between the two variants. The subsequently reported results concerning H0 were corrected to control false discovery by Benjamini-Hochberg procedure [25] as multiple pairwise tests were conducted. Nonetheless, following McDonald [26], the raw p-values of the remaining significant results are stated to indicate authentic probabilities. In none of the assessed items was the paper-based ideation valued as more supportive than the digital representation.

Fig. 7.
figure 7

The digital cards were perceived as more helpful in finetuning existing ideas

Full size image

Ideation.

When first analyzing the ideation support results, the groups co-creating with the remote tool perceived the digitized cards as more helpful for finetuning ideas than the groups using the paper-based version (Fig. 7). A majority of 75% of the students using the remote tool agreed on support through the digital cards, U = 634.5, z = 2.83, p = .005, r = .36. Contrarily, only 39% of the students in the co-located group were experiencing help for improving existing ideas from the paper cards.

Balance.

Looking next at the support for balancing the game concept, the technology-enhanced CGF co-creation activity was perceived in total as significantly more supportive (Fig. 8). While 46% of the participants considered the paper-based toolset in general as helpful for balancing the game concept parts (playing, teaching, researching, and designing), 80% felt supported by the remote co-creation tool, U = 612.5, z = 2.52, p = .012, r = .32. In detail, participants expressed more perceived balancing support from every component of the digitized toolset. The remote co-creation groups felt more support from the digital cards for balancing the concept between the parts than the groups using paper cards, U = 653, z = 2.81, p = .005, r = .36.

Fig. 8.
figure 8

The digitized tool was found significantly more helpful for balancing the game concept

Full size image

Similarly, the paper board was experienced as significantly less helpful than the digital playboard for balancing the privacy education goal to the game goal, U = 623.5, z = 2.36, p = .018, r = .30. The groups felt particularly differently about help for balancing originating from the roles (Fig. 9).

Fig. 9.
figure 9

Digital implementation of the role tokens was perceived as significantly more supportive

Full size image

The remotely collaborating groups perceived significantly more support from the digital role representation for balancing the game parts, U = 634, z = 2.5, p = .012, r = .32, balancing the educational goal to the game goal, U = 659.5, z = 2.88, p = .004, r = .37, and for identifying conflicts in the educational game concept, U = 649, z = 2.73, p = .006, r = .35.

Fig. 10.
figure 10

The digital co-creation was perceived as significantly more applicable and enjoyable

Full size image

Application.

When analyzing the two toolset variants’ applicability, three significant differences between the toolsets were identified (Fig. 10). First, remote co-creation with the digital CGF was perceived as more fun than the paper-based idea finding sessions, U = 610.5, z = 2.19, p = .029, r = .28. Second, students were expressing more confidence to use the digital remote co-creation tool on their own than the co-located paper toolset, U = 599.5, z = 1.99, p = .047, r = .25. Third, the remotely collaborating groups also felt more satisfied with their co-created privacy education game concepts than their colleagues using the paper-based toolset, U = 626, z = 2.41, p = .016, r = .31.

The median (Mdn) value for all the above-reported results was 5 on the Likert scale regarding the remote co-creation and 4 for the co-located variant. The effect sizes were hovering around r = 0.3 and thus indicating medium effect sizes for all findings [27]. Statistical analysis consequently suggests rejecting H0 as support from remote and co-located toolsets was perceived significantly different by the student groups.

Co-created Game Concepts.

Finally, the co-created outcome was analyzed regarding the most frequent game design choices. Table 1 lists the design suggestions per card deck that were selected more than once.

Table 1. Student groups’ preferred design choices for analytic privacy education games
Full size table

4 Discussion

4.1 Supporting Remote Co-creation with the Digitized Card/Board Metaphor

When looking at the results with respect to the first research question, the study disclosed that the technology-enhanced card/playboard metaphor considerably supports groups with idea finetuning and concept balancing. In fact, remote co-creation with the digitized toolset was perceived as more supportive, applicable and enjoyable than the paper-based co-located ideation card activity. The feeling for more support thereby originated from all investigated parts of the digitized toolset.

First, concerning ideation support, the digital cards were found more helpful in finetuning existing ideas. However, it was the only significant difference between the groups regarding ideation. From the study findings, the digitized card metaphor as designed in this toolset comparison can be seen as equivalent for idea finding to paper-based cards but more supportive for idea improvement. The benefits for idea finetuning with the digital cards could result from an improved speed to try out combinations in the game concept. Compared under Fitts’ law [28], placing cards in the slots on the screen with drag-and-drop support is more efficient than laying out cards on a big A0 paper. This is further supported by the fact that the digital cards were found more supportive for balancing between the game concept parts.

Second, balancing support was found to be significantly better with the digital than with the paper tool. While the digital cards were found as helpful to balance between the game parts, the board supported matching of educational goal to game goal. As the playboard was featuring very similar areas and layout in both versions, the digital playboard’s advantage likely originates from the more guided approach. The synchronizing effects of time countdown and introductory voice instructions before each step are guiding structures that could support the balancing process as suggested by previous research [20, 21]. However, the most improvement for balancing was found with the digital representation of the roles. The inclusion of emotional feedback for each role in the remote co-creation scenario can be one factor contributing to this (Fig. 4). It represents a quality goal [21] that is both an emotional cue and a measure of each role’s current satisfaction with the overall concept that can help maintain discussion between the co-creators. Consequently, discussing discrepancies in the satisfaction rating may support finding conflicts in the concept.

Finally, regarding application of the toolsets, it is remarkable that the digital co-creation in the remote setting was found as the more enjoyable activity that creates a more satisfying game concept outcome. Even more remarkably, about 70% of students would use the digital toolset independently, compared to only 40% for the paper-based option. The reasons for this might be found in the same digital design factors that are behind the improved ideation/balance support. The more guided approach can be a factor that contributes to this higher applicability. In the same way, the emotional feedback for the roles can add to a more engaging discussion and, as a result, to a more satisfying co-creation outcome.

Conclusive synthesis of the results provides design implications for technology-enhanced co-creation card toolsets:

  • The digital card/board metaphor with drag-and-drop interaction supports ideation in remote settings comparable to co-located paper-based tools and helps improving ideas.

  • Feedback options that allow to emotionally rate the overall concept at any time support balancing between role-oriented concept parts.

  • Guiding mechanisms such as a stepwise introduction before each phase and step countdowns support applicability and concept balancing during remote co-creation.

4.2 Co-created Outcome of Preferred Privacy Education Game Designs

Concerning the second research question, several patterns emerged through frequency analysis of the co-created game concepts (Table 1). In general, the outcomes of both card-based co-design activities display similar design choice patterns. The results suggest that co-located and remote card-based co-creation can be employed complementary or interchangeably for designing educational game concepts. However, further research with co-creation sessions including students with more diverse backgrounds is required to confirm this implication. When looking at the combined frequency patterns, students revealed a preference of choosing themselves as the games target group and building a game for continual voluntary/free play at home. A clear alternative pattern, however, was the university/school context. Selected privacy challenges showed no apparent preferences but addressed movement tracking, data aggregation, data security knowledge and privacy trade-off behavior.

Design choices for the player role showed preferences for competitive, mission-oriented games that progress by scoring and adapting by increasing rewards/difficulty or introducing new elements. In terms of the teacher parts, most groups suggested encouraging reflection about privacy choices by displaying retrospective decision trees/summaries and examining progress by tracking time/movement patterns or asking subtle questions by characters. Looking at the researcher role, most of the students chose to research risk-taking or decision-making. The groups most frequently suggested investigating these objectives with monitoring of events, team decisions or interaction patterns while asking story integrated questions by characters or including micro questions/experience sampling. No strong preferences could be identified for the designer perspective. Game presentation choices ranged from Augmented Reality to map/bird perspectives or other 2D presentations. However, most groups chose to design games for classical point-and-click or tap interaction.

5 Conclusion

The performed study outlined the digitalization of a paper-based card ideation tool and applied it in a remote co-creation scenario to investigate applicability and user support compared to using the paper tools in the classroom. The between-subjects trial displayed that co-creation with the digital card/playboard metaphor in a video conference session is equally supportive for idea generation than using co-located paper tools and significantly more helpful to improve ideas and balance concepts. In addition, the results showed that integrating guiding/feedback mechanisms that become available for a digitized toolset improves balancing support and applicability. Providing options to rate the concept freely and narrated stepwise instructions showed to increase students’ satisfaction with the outcome, enjoyment of co-creation and confidence to independently use the digitized tool.

The design patterns in the co-created privacy education game concepts indicate that the remote and co-created co-design resulted in comparable game concepts. Students showed a preference for competitive, mission-oriented games that progress through scoring, taking turns, or solving mini-quests and adapting to player skill by introducing new rewards and increasing difficulty. Students also showed a preference to integrate decision summaries about privacy choices for reflection and examine progress by logging and subtle in-game questions. Regarding privacy research factors, the student groups preferred to investigate risk-taking or decision-making by equally unobtrusive strategies. Future research should extend on these favored game design strategies when creating privacy education games. As the digitized card toolset has demonstrated applicability and user support in this study, upcoming research endeavors are encouraged to research and improve this utility further by dedicated co-creation sessions, including educators and researchers.