1 Introduction

Augmented Reality (AR) and Game-Based Learning (GBL) are two approaches that have been used by researchers and developers in recent years to provide enhanced learning experiences (Fotaris et al., 2017; Pellas et al., 2019). On the one hand, AR is the superimposing of a virtual layer of information over the real world. Given the characteristics of AR technology, when it is integrated into the learning process it boosts the learning experience by creating many different learning benefits and opportunities (Bacca et al., 2014; Diegmann et al., 2015).

On the other hand, we consider GBL as the use of games with learning aims, considering a game for learning as defined by Steinkuehler and Squire (2014, p. 11), who based their definition on Thai et al (2009) as “…a voluntary activity structured by rules, with a defined outcome and feedback (e.g., points) that facilitates reliable comparisons of in-player performances…[that] target the acquisition of knowledge as its own end…”. GBL is based on the fact that games have certain characteristics that allow for “playful learning” (Klopfer et al., 2009) which in turn offers benefits such as improving motivation, encouraging learning from failure, supporting problem-solving among others (Gee, 2008; Klopfer et al., 2009).

In this study we consider that the next innovative step in the use of these technologies has come through the amalgamation of both approaches. This union, which will be referred to simply as Augmented Reality Game-Based Learning (ARGBL) (Fotaris et al., 2017; Pellas et al., 2019; Tobar-Muñoz et al., 2015, 2017), is defined as using games for learning by applying AR technologies. ARGBL seems to be a natural development because AR learning experiences may benefit from the more interactive and problem-based approach that games for learning have, and games for learning may benefit from the augmented experiences that enhance the real world. In the same way, GBL proposes that students benefit from many properties such as the emotional and motivational aspects of achieving in-game goals, the rewarding sense, the structuring of the activity via rules that can be used by students in order to learn and perform, among other advantages such as those listed by Gee in his work (Gee, 2005). On the other hand, AR is an immersive technology that allows students to experience by first hand visual aspects of learning objects and experiment with them in their own context using 3D and multimedia capabilities of mobile and desktop devices (Diegmann et al., 2015). For instance, an ARGBL in a biology class could be a 3D game with the AR visualization of an animal cell with the goal of feeding it and making it move by using the cell’s organelles functions in a videogame fashion.

In order to bring these experiences to the classroom, more and more, they have to be created and implemented. However, currently there is a gap between the developers, who would like to bring these experiences to the classroom, the teachers, who are not aware of the characteristics of such experiences, and the students, who would be the final users. This gap has to be closed to successfully bring educational innovations, such as ARGBL games, to the classroom. This gap enacted the following research question which is addressed in this paper:

RQ

What guidelines should Teachers and Designers follow in order to create a motivating and pedagogically effective ARGBL experience?

In this paper, a method for the Co-Design of ARGBL experiences is described, applied, and validated. In the section “background”, we outline the background of studies, paradigms and other frameworks which were considered before we proposed our method. Next, the method we developed is briefly described, followed by the two case studies where the method was applied are described. Then, the validation of the method is shown and, finally, we show the discussion, conclusions, and future work of this study.

2 Background

Interest in the field has been growing even to the extent that many systematic reviews have been conducted exploring the use of ARGBL. The study conducted by Weerasinghe et al (2019) studied how 30 ARGBL games were used in studies from 2006 to 2017 linking their properties to learning theories such as constructivism and humanism with positives outcomes. Furthermore, Tobar-Muñoz et al (2017) conducted a review analyzing 27 instances of ARGBL in many disciplines (mainly on Science learning) and mostly going from elementary education to early upper education. It is also the case for Yu et al (2022) who analyzed 46 studies using ARGBL games for STEM education. And finally, Alper et al (2021) analyzed 53 scientific studies involving the use of ARGBL experiences, also mostly on STEM areas but also for subjects such as History and Native Languages.

Recently, several studies have reported ARGBL experiences with many types of AR and games (Catal et al., 2020; Fotaris et al., 2017; Koutromanos et al., 2015; Laine, 2018; Laine et al., 2016; López-Faican & Jaen, 2020; Nuñez et al., 2008; Ortiz et al., 2018; Tobar-Muñoz et al., 2017) which in many cases show benefits such as improved collaboration and learning retention.

Some studies have reported ARGBL experiences with many types of AR and games (Tobar-Muñoz et al., 2017). For example, ARGBL has been demonstrated through Geo-Located AR (Dunleavy et al., 2008; Klopfer & Sheldon, 2010; Klopfer & Squire, 2008; Rosenbaum et al., 2007; Squire, 2010; Squire & Jan, 2007), Marker-Based Augmented Reality (C.-H. Chen et al., 2015; Gomes et al., 2014; Guenaga et al., 2014; Lin et al., 2011; Marco et al., 2009; Tobar-Muñoz et al., 2014) Image-based (Chen & Chan, 2019), and QR-Code-based AR (Bressler & Bodzin, 2013). Also, ARGBL is an opportunity to propose learning activities using board games such as the one proposed by Lin and Hou (2022) and the one by Li et al (2018), demonstrating how such games improve learning and performance while lowering learning anxiety.

ARGBL has gained attention from the scientific community and most studies suggest that design guidelines should be followed in order to achieve learning and a good implementation in the learning processes. For example, Weerasinghe et al. (2019) proposed four design guidelines that advise designers on aspects such as feedback, collaboration, elements for the real world and modelling the real phenomena. Also, Li et al (2022) proposed eight design guidelines as well, increasing the need for designers to consider them while working on their ARGBL games. These design guidelines suggest that designers must be concerned on the particularities of ARGBL games to propose compelling, playful, and educational ARGBL games, while teachers are the ones capable of understanding the concepts and the subject matter to be taught. Thus, a co-creation method is timely to fill the open issue of relating teachers and professional ARGBL designers.

In this paper, an ARGBL co-creation method is proposed and validated; this method is called Co-CreARGBL. Co-CreARGBL is based on the paradigm of Co-Design for Learning, which is mainly interested in designing new artifacts, in particular, technologies for learning. This paradigm of design is especially interested in involving users, not only those who “lead” or are “creative”, but anyone who is inspired and passionate about participatory design techniques. Consequently, Co-Design for Learning is used to create educational artifacts and innovations. As defined by Penuel et al., (2007, p. 53), Co-Design for Learning is a: “highly-facilitated, team-based process in which teachers, researchers, and developers work together in defined roles to design an educational innovation, realize the design in one or more prototypes, and evaluate each prototype’s significance for addressing a concrete educational need”.

While teachers have already been employing Co-Design when creating learning games, their experiences to date refer to collaborating with their peers and using an authoring tool but not actually working with professional designers (Frossard, 2013). The method proposed here emphasizes the need to include both teachers and designers in the design process. Teachers contribute with their knowledge of their educational environment and designers and developers build the actual artifact.

This study argues for the inclusion of teachers in the process of design as a number of researchers in AR for learning consider that creating AR experiences greatly benefits from collaboration between designers and teachers (Cuendet et al., 2013; Santos et al., 2014). Some new experiences have entered the Co-creation and Augmented Reality for Learning arena such as the work of Bacca et al. () who describes an application, which incorporated inclusive features to support learning in Vocational Education and Training (VET) processes.

As it can be seen, ARGBL experiences have shown to be useful for several learning processes, thus an approach for creating this kind of experiences is of importance.

3 Co-CreARGBL Method

Co-CreARGBL is a co-design method intended to create ARGBL experiences including the expertise of both, teachers, and designers. Co-CreARGBL has some general characteristics which help to picture what the domain of application of the method is and what its approach is. Co-CreARGBL is a method to guide creative design processes of craftsmanship aimed to create learning artifacts. It is also applied to long processes of design, mediated by dialogue among stakeholders, and guided by design principles.

Co-CreARGBL proposes the use of a set of roles, which have been defined using the high-level approach proposed by the Six-Facets model (Marne et al., 2012). The roles are:

  • The Leader: who leads and coordinate.

  • The Designers: who design the game and “scaffold” teacher’s ideas.

  • The Developers: who construct the game software and assets.

  • The Researchers: who are involved in the analysis of the game and its use.

  • The Teachers: who know about the content and the educational context.

  • The Students: who play. It is desirable for students to participate in the design process.

Co-CreARGBL is a three-stage process: training, iterative design, and classroom evaluation. Each stage has a set of suggested activities to be included in the process. Each activity has a set of considerations to be taken while conducting the process.

  • Training: During this stage, teachers get familiarized with the potential of ARGBL.

  • Iterative Design: Here, the team participates by ideating, designing, and developing the ARGBL game. The participants work on a series of activities and tasks under the structure of the first part of the SADDIE process: Specification, Analysis, Design and Development (Rugelj, 2015; Zapušek & Rugelj, 2014) including the relevant activities for AR (augmented elements) and GBL (rules, goals, scenarios).

  • Evaluation: Here, the team tests the behavior of the product in a learning environment and involves deploying the product and observations on the development of the learning and playing activity as discussed by Carvalho et al (2015).

4 Applying the Method—Two Case Studies

In order to validate the method, two case studies were conducted. The paradigm applied to the validation process was Design-Based Research (DBR). DBR is “…a series of approaches, with the intent of producing new theories, artifacts, and practices that account for and potentially impact learning and teaching in naturalistic settings.” (Barab & Squire, 2004). With this in mind, a series of interviews were carried out with AR and GBL professional designers and teachers to audit important educational aspects and how ARGBL could impact teaching scenarios. This led to the idea of constructing a method to help them develop ARGBL experiences collaboratively. In the light of this, a conjecture map –a way to describe how the researchers believe a practice can help the educational setting– was proposed (Sandoval, 2014). Table 1 summarizes the conjecture map.

Table 1 Conjecture map developed for the research
Full size table

The conjecture map was used to describe the aims of the design experiment. Design iterations helped to refine the method with input from participating Teachers, Researchers and Designers.

The two case studies comprised two teams of teachers working along AR and GBL designers. Bellow, a summary of the participants and their roles in the Co-CreARGBL process is shown.

  • Team A: 4 Teachers, 2 Leaders, 1 Designer, 1 Researcher, 2 Developers. Worked on an ARGBL game on the subjects of Geography and Social Sciences

  • Team B: 2 Teachers, 2 Leaders, 1 Designer, 4 Developers. Worked on an ARGBL game about Mother Tongue and Cultural Philosophy.

Each team shows the subject chosen by the Teachers. Teachers were aged varying from 32 to 45 years old. All of them were public-school teachers. None of the teachers had experience with AR and/or GBL.

In the training sessions, the Teachers experimented with AR authoring tools and AR applications from the Google Play Store, as well as a variety of games for learning. Teachers were also introduced to the ATMSG model by Carvalho et al. (2015) and the library it proposes. This model is proposed as guide to follow when creating Serious Games, so it was used as a basis to propose game mechanics during the iterative design stage.

Teachers in Team B work for a school belonging to the Nasa indigenous community in the south-west of Colombia. Teachers agreed to work on a learning objective that could be applied to the subjects belonging to the indigenous culture. Thus, they decided to work on Mother Tongue which refers to the learning of the Nasa Yuwe (the indigenous culture language) and the learning of their traditions.

During the Design Activity, weekly design meetings focused on designing specific parts of the games. Here, Teachers and Designers listed the main augmentable elements that could be found in a classroom, from among which they selected the maps of the Cauca Department for team A and traditional symbols and calendar for Team B. Other augmentable elements included a set of markers for interaction and information retrieval concerning the game elements using relevant imagery chosen by the Teachers. The Teachers and Designers built a small set of paper prototypes (Fullerton, 2008) which consisted mainly on printing the markers-to-be images and specifying some rule testing for the team. When a prototype idea was accepted by the team, its details were then included in a set of design documents. Both teams devised a sub-system in the game allowing the students to retrieve information about the educational content.

4.1 Team A’s Resulting ARGBL Game

This game was an educational Videogame with AR whose Learning Objective was “to identify the diversity and richness, (geographical, ecological, and historical) for locals and tourists alike of the Cauca Department”. This game, called “Una Aventura por el Cauca” (“An Adventure around Cauca”), is displayed on a mobile device pointing towards a marker-referenced map (Fig. 1a). The game’s main means of interaction is via a set of AR markers printed with icons depicting their intended use (Fig. 1b). In the game, the Students must act as tourist guides whose mission is to lead the “invisible” visitors (who become visible through the mobile device) to their desired destinations according to their requirements (Fig. 1c). To this end, Students familiarized themselves with the characteristics of each municipality through an information system designed to give facts about the specific municipality (the learning content) and other game-related information (Fig. 1d).

Fig. 1
figure 1

Team A resulting product "An Adventure around Cauca": An ARGBL game created through Co-CreARGBL

Full size image

During the Implementation Activity of the Classroom Evaluation stage, the Leaders and Teachers found a group of schools where they could test the game. The Teachers designed an instructional activity for the classes, which included the game and a script in keeping with the ATMSG model, Carvalho et al. (2015). During the Evaluation Activity stage, the game was tested in two scenarios. These scenarios were natural because they comprised a set of 15 and 30 students who interacted with their teachers and their regular companions in their natural classroom setting as opposed to a laboratory setting. These settings are shown in Fig. 2.

Fig. 2
figure 2

Teacher and Students using “An adventure around Cauca” in a naturalistic environment

Full size image

4.2 Team B’ Resulting ARGBL Game

Team B’s is a board game with AR (Fig. 3a). It is a Mobile Video-Displayed Marker-Based AR game to be used with an Android tablet or smartphone. The game works by pointing the camera at a game board with a set of marker pieces, each of them represent the elements of the game. The AR system implemented within the software application allows the game to work as an information source which also incorporates the educational content.

Fig. 3
figure 3

Team B resulting product "Cuetaya: Land of Colors" an ARGBL game created with Co-CreARGBL

Full size image

The game was called “Cuetaya: Land of Colors” and its goal is to marry the two characters: Sek (Sun) and Çxayu’Çe (Happiness) (Fig. 3b) under the traditional values of the Nasa culture. The game can also be used to learn about the Nasa calendar known as A’te Dxi’J (the path of the moon). The calendar is not only printed on the game board, but is also an interactive augmented calendar visible through the AR device, providing relevant information (Fig. 3c). The game has a marketplace where items can be exchanged for corn and other materials such as wool, wood and other construction elements (Fig. 3d).

The Evaluation Activity was carried out and evaluated in the indigenous school where the Teachers participated in the project worked. The evaluation activity was conducted with 13 Students aged between 8 and 10 years old.

Figure 4 shows the naturalistic environment setting where Teachers and Students used the ARGBL game.

Fig. 4
figure 4

Teacher and Students using “Cuetaya: Land of Colors” in a naturalistic environment

Full size image

5 Validating the Method Using the Two Case Studies

The Validation Square (VS) framework (Seepersad et al., 2006) was used to validate the method. The VS framework is used to validate design methods and considers the validation of a method as the process of “… building confidence in the usefulness [of the method] with respect to a purpose” (Seepersad et al., 2006). Unlike other traditional validation processes, it considers validation in a holistic and semiformal way which, as its main authors and other adherents consider, is appropriate to validate design methods that involve aspects difficult to measure and more subjective considerations. Next, a set of Acceptance Conditions (AC), suggested by VS, are shown.

5.1 AC1 Accepting Parent Constructs’ Validity

Construct’s validity refers to the validity of the theories, recommendations, and structures in which the design method is supported. We argue that Co-CreARGBL uses stages, roles, activities and recommendations already supported by AR and GBL literature.Table 2 shows the relationship between the constructs of Co-CreARGBL, their corresponding parent construct, and how they support the method.

Table 2 Summary of the Co-CreARGBL constructs and their relationship to their parent constructs
Full size table

5.2 AC2 Accepting Internal Consistency

Internal Consistency refers to the way the constructs are assembled in the method. Accepting the consistency means accepting that, for each step in the method, there is adequate input and output (Seepersad et al., 2006). Figure 5 shows the flow diagram depicting the flow of information as proposed by Co-CreARGBL.

Fig. 5
figure 5

Diagram of information flow in Co-CreARGBL

Full size image

The flow diagram shows how, from the beginning, each step fuels the subsequent steps. It shows that, initially, there is a set of tasks that must be carried out before selecting the teachers who participate in the training stage and Leaders contact schools.

Then, by the end of Training stage, Teachers are expected to have been trained in the Design Models (DM) and the Principles (Ps) to be used during the design. These principles, as has been noted in the description of the method, guide the process for creating games and AR applications. When the Iterative Process starts, the Teachers (Ts) and the Designers (Ds) who will participate in the process are selected. Then, the SADDIE process begins. The Specification activity should define the Learning Objectives (LO), the Educational Content (ECo) and (optionally) the Evaluation Criteria (ECr) in a Specification Document (SD). Note how those elements are being used afterwards in the process. LOs are used during the Design stage to define the Game Objectives (GO) and these are used to define the Augmentable Objects (AO) that will form part of the ARGBL game. The Analysis activity analyses the elements defined in the SD and creates an ARGBL game idea (perhaps using the framework that Co-CreARGBL proposes) and produces an Analysis Document (AD). Then, the design process defines the rules, mechanics and AOs, interactions, and information, based on an iterative process that, when necessary, returns to the activities in the Specification and Analysis stage.

The elements defined during the Design activity then fuel the Development activity which, in turn, produces a prototype to be used during the Classroom Evaluation stage. However, note that Students are expected to be in touch with the Digital and Paper Prototypes as they help testing the mechanics and giving ideas. The activities in the last stage utilize the prototype and define a Learning Activity and Materials based on the ECo and the ARGBL game. During these activities, an optional Evaluation Instrument can be designed based on the ECr. From there, there can be a reiteration if needed, looping back to the Iterative Design stage.

5.3 AC3 Accepting the Adequacy of the Example Problems

Table 3 shows the various characteristics for which the method is intended and the accomplishment by the example problems.

Table 3 Critical characteristics of Co-CreARGBL compared against the characteristics of the example case studies
Full size table

5.4 AC4 Accepting the Usefulness of the Method for the Example Problems

Usefulness of the method, for Co-CreARGBL, was considered to be the quality of the products which in turn was assessed as the increase on Motivation and the observed Learning Gains.

For both teams, Motivation was assessed using the Intrinsic Motivation Inventory (IMI) (Ryan, 2006) with a Likert scale using smiley faces as suggested by Mellor and Moore (2014). The sub-scales used in this study were assessed as Likert Scales and they included Interest/Enjoyment (seven questions), Effort/Importance (five questions), and Value/Utility (seven questions).

For Team A, the trial took place in a natural classroom environment with 25 students (77% male and 23% female) aged between 8 and 10 years old (M = 9.07, SD = 0.75). Team B’s trial was also conducted in another naturalistic environment with 14 students (55% male and 45% female) aged between 9 and 11 years old (M = 10.00, SD = 0.86).

In both teams Teachers and Researchers were able to perform a comparative study against some traditional learning material (i.e. an Info Sheet in the form of a website considering the same educational content as the game) that teachers would use in this class. Thus, the IMI included, for each question, an answer for the Info Sheet and an answer for the ARGBL game.

The analysis of motivation report using the IMI then was done by assigning a score to each of the Likert scale items (1 to 5 points). The scores were averaged for each sub-scale. A t-Test performed on each pair of assessments resulted in a significant difference, one which saw the ARGBL game favored over the Info Sheet (p-value < 0.05 for both teams). The graph Fig. 6 shows the final results. These results suggest that, judging by the students’ self-report, students were more motivated using the game than the Info Sheet. This is natural, as the game is a novel and fun way of transmitting the same contents included in the Info Sheet.

Fig. 6
figure 6

Results from the Motivation assessment of both teams’ games against the Info Sheets using the IMI

Full size image

As for Learning Gains in Team A, the Teachers designed an Evaluation Instrument based on the themes which formed parts of the Learning Objectives and were labeled as: Tourism, Festivals, Hydrology, Archeology, Economy and General Aspects of the Cauca Department. The Evaluation Instrument consisted of a set of simple open questions on the themes. The assessment consisted of asking the Students a set of questions before they played the game (Diagnostic Evaluation) and then again after the game (Formative Evaluation). In total, 25 students were analyzed.

For each Student, Teachers scored their answers on a scoring sheet, using a rubric designed together with the Evaluation Instrument. The rubric gave the Students’ answers a score of 1 to 10. Figure 7 shows the Students’ scores during the diagnostic and formative evaluation for each of the themes defined in the Evaluation Instrument.

Fig. 7
figure 7

Student scores during Team A’s Classroom Evaluation

Full size image

A Wilcoxon matched-pairs signed-ranks test conducted on the data the scores provided, demonstrated a significant increase in the scores (N = 25, T = 25, p-value < 0.05).

For Learning Gains in Team B, the Teachers planned a Diagnostic Dialogue to be conducted before the learning activity using the ARGBL game. A diagnostic text for each of the students was generated based on the documentation (Diagnostic Evaluation) and with the help of the Teachers, to assess the Students’ previous knowledge of the themes planned for the learning session. Then, Teachers conducted an evaluation activity consisting of asking the children to draw picture of what they had learned after playing the ARGBL game. After the evaluation activity the Teacher wrote an evaluation report (Formative Evaluation) for each of the Students and a “Final Observations” text.

All the evaluation texts (Diagnostic, Formative and Final Observations) were analyzed by two independent researchers who agreed on a set of labels to be used to code each of the evaluation texts. Coders agreed to use the coding based on an agreed formula: < Category > : < Specification > : < Theme > . The category part of the code was used to define the aspect that was being observed in the students’ answers or in the evaluative texts such as the student’s “Previous Knowledge” or whether a “Difficulty or Lack” had been identified. The specification part of the code was used to determine the aspect evaluated such as “Recognition of a Concept” for the Previous Knowledge category and the Theme was a free word that described what the evaluation was about.

The evaluation was conducted by the coders to try and find the learning gains of the students according to the judgment of the Teacher. The two researchers who coded the evaluation text agreed 68.57% of the time. A final coding was agreed through mediation which sought consensus.

  • Diagnostic Evaluation

    1. o

      Previous Knowledge: Associates: Nature and Family (15), Considers themselves: Nasa Being (15), Recognizes: Nature items (14), Recognizes: Nasa Yuwe (7), Recognizes: Nasa Rituals (4), Recognizes: Uma and Tay (3)

    2. o

      Difficulty/Lack: Does not Recognize: Nasa Calendar (15), Does not Recognize: Moon Phases (14), Does not Recognize: Nasa Rituals (8)

  • Formative Evaluation

    1. o

      Learning: Gives Importance to: Nature (7), Gives Importance to: Construction of the House (3), Gives Importance to: Family Values (3)

    2. o

      Difficulty/Lack: (none)

  • Final Observations

    1. o

      Positive Outcome: Nasa Rituals (7), Nasa Calendar (6), Moon Phases (8), Traditional symbols (3), Nasa Yuwe (3), Maize Crops (3), Traditional Barter (3)

    2. o

      Negative Outcome: (none)

As a final conclusion, judging by the coding of the evaluation texts, Students seem to have acquired basic knowledge on the themes purported by the Teachers. Perhaps, the use of the ARGBL experience would have resulted in clearer results had the themes been addressed directly by the teacher and not as part of a broader class. On the bright side, the activity had a positive impact on the students by helping them to recognize the basic aspects of the Nasa traditions and family values. On the bright side, the activity had a positive impact on the students by helping them to recognize the basic aspects of the Nasa traditions and family values. Future applications of the ARGBL experience and more in-depth studies are needed to conclude the final impact the game has.

5.5 AC5 Accepting Usefulness is Linked to the Method

A checklist of tasks was prepared and verified according to the activities performed during the Co-Design process. This resulted in the number of activities suggested by Co-CreARGBL that were performed by each team.

A total of 123 tasks were planned, and while some of them were transversal (i.e. to be applied to most of the activities throughout the process) they were assigned to each activity where they were relevant. The completion of the tasks resulted in 88% completion by Team A and 82% by Team B.

Moreover, to gather information on teacher comprehension of the process, a set of three interviews were conducted with the Teachers who acted as designers. The first interview was immediately after the start of the process, another followed the Development activity, and the final one was held once the process had been finalized. A Thematic Analysis was conducted on the recorded interviews. Next, the most relevant themes regarding Teacher comprehension are described.

5.5.1 Comprehension of the Training Stage.

Although Teachers do not use formal terms, their answers show that they understand and value the importance of the Training stage, which they called “capacitación” (a process of ongoing training). The following are excerpts from the interview:

[The training stage] is a strategy that helps… because it is hard to unlearn stuff you come with…

[The training stage] is important, because it tells them [teachers] what the uses of a learning game are

Teachers considered that the design is supposed to be guided by a set of principles. However, Teachers were unable to recite them later; however, in their own words they mention principles. For example, they mentioned: Understanding, innovation, motivation, fun, analysis of educational content, avoid behaviorism, usability, didactics, play, stealth learning, and attractiveness, among others.

5.5.2 Comprehension of the Iterative Design Stage

Teachers’ answers here demonstrate that they have a general understanding of this stage. They appreciate the process that was being conducted as an iterative process of design. Teachers value positively the process during their discourse. They thought it was important that the process was iterative, collaborative, and interdisciplinary.

I think above all there is the design… before moving on to the game… The main thing is to design what we want to do, specify the learning object and then the game’s objective, which are quite different

When we proposed the contributions then we saw whether they would work or fail… If the idea was not pedagogical, or it wasn’t aligned with the learning objectives then we adjusted it to modify it…

Based on the information that was given (the educational content) they [the designers] created the design… but when we saw flaws or inconsistencies, we improved it.

5.5.3 Comprehension of the Evaluation in Classroom Stage

Teachers were asked to give a brief step-by-step outline of this stage. In their answers, they also mentioned the tasks involved in classroom evaluation: correct the design, observe the learning outcomes, implement the game with students in the classroom, observe how students receive the game, and evaluate the capacity of the game to reach the learning objective, among others.

Teachers explained that during this activity the school was supportive and helped by allowing the Students to be evaluated and the research to be carried out in the school. One Teacher explains that, “… the school helped us when we explained to them that we were bringing an Augmented Reality tool and it was innovation for our school…”.

In general, the thematic analysis suggests that Teachers comprehend the method they were following and its objectives, and they were able to grasp some concepts pertaining to the Co-Design of ARGBL games and experiences. Nonetheless, while they have a general understanding, they are not experts in the field, nor are they required to be.

5.6 AC6 Accepting the Usefulness of the Method Beyond Examples

According to the VS, usefulness beyond the examples is validates through a set of arguments in favor of the use of the method based on previous AC’s (Seepersad et al., 2006).

On one hand, the method is consistent with other methodologies for building Learning Resources with Games and with Augmented Reality. The method is in many ways a restructuring of those constructs and it considers many important aspects (such as Augmentable Objects) of designing and developing learning resources using games and Augmented Reality. On the other hand, the method has demonstrated internal consistency by showing how the different stages and activities complement each other by offering the right information and requiring the actors in the process to produce information adequate for the AR game.

Two case studies were used to validate the method in real scenarios. These case studies have been analyzed and validated from a theoretical and performance point of view. With the arguments mentioned in the previous sections, the case studies prove adequate as they classify any problems where the method is to be used as acceptable. The method was useful for the two cases and its usefulness seems to be linked to applying the method. Moreover, the teams in the case studies followed the tasks that were planned following Co-CreARGBL which suggests that the usefulness of the results is linked to the method.

Note that, the case studies in which the method was applied involved teams with little to no funding at all and Designers and Teachers participated voluntarily. Thus, while the method is intended to be used with Teachers who do not have access to full funding and simple AR, better results are to be expected if the process were to be applied to a bigger, better-funded project.

It is noteworthy that the theoretical performance of the method greatly depends on the expertise of the Designers, the willingness of the Teachers and the capacity of the Leaders to conduct the process. Note also, that the method is intended for long-term projects that span months, not short-term projects and/or simple quiz-like or authored games and this should be considered.

6 Discussion

We argue that ARGBL experiences for learning harness the potential of analog and digital gaming offering students with compelling experiences which can be used to both socialize and learn while motivation students. Also, we show that an effective way to create ARGBL games and experiences is to follow a method that guides the work of teachers and professionals’ game and AR developers through a Co-Design approach. As the experiences described here show, following this approach is effective as the teams described here were able to create two ARGBL experiences and take them into the classroom. This was a collaborative effort where teachers participated as designers and other professionals were in charge of a building the ARGBL game; thus, the teachers were not encumbered with the burden of having to build the ARGBL game.

Furthermore, in the literature review conducted there were some constructs such as design models, frameworks and recommendations for creating AR applications for learning. These constructs were considered in the design of the proposed method and include the uses of AR that Santos et al., (2014) report in their studies, the affordances of Cheng and Tsai (2012), and the directions of AR in education by Yuen et al. (2011). These constructs are very similar to the ones for GBL design i.e. often very high level and deal only with general aspects of the AR design of applications. Again, the method proposed here is a consolidation of these constructs to combine AR applications with games and render them useful for the classroom.

In the past, other approaches have been proposed for creating AR and/or games for learning, especially authoring tools (Eldokhny & Drwish, 2021; Frossard, 2013; Lim & Lee, 2013; Lytridis, 2018; Marfisi-Schottman et al., 2010; Mehm et al., 2012; Mellini et al., 2011; Proactive, 2011). However, authoring tools, by nature, create simpler AR or GBL experiences, because they are made for non-expert users such as the Teachers. The Co-Design approach of Co-CreARGBL is different in that it advocates many people from different backgrounds participating in a collaborative effort, which in time allows for more complex experiences. However, this brings to the fore the issue that, while authoring tools may be used in days or weeks, a Co-Design process may take weeks or months, something which may not be desirable for some teachers or institutions. The advantage to Co-Design approaches such as the Co-CreARGBL method is that the complex experiences created can be utilized in other environments such as other subject classes, schools or playgrounds, thus fully exploiting the extra time required to create them. To the extend of our knowledge, no method to relate Teachers and Professional Designers has been proposed, explored, and validated as most studies have focused on creating simple experiences by Teachers. Furthermore, similar methodological approaches to create educational resources have been proposed in the past. However, these approaches are often either too general or do not consider aspects like Co-Design or the particular considerations of AR and/or GBL. Co-CreARGBL is an approach that taps into the particular aspects of ARGBL and aims to be a useful repository of ARGBL principles, properties, and other important considerations in ARGBL design.

The Validation Square (Seepersad et al., 2006) was used as the validation framework for Co-CreARGBL because of the characteristics of its process. This framework was chosen because it considers the validation of the method as a process of building confidence on its usefulness, and it understands usefulness as whether the method provides design solutions correctly and with acceptable operational performances. We argue that this style of validation is the most adequate for a method such as Co-CreARGBL because it does not focus on a strict, objective, or mathematical validation, but rather on a holistic one that considers the validity of the method from several perspectives, all of which have been presented to build confidence in Co-CreARGBL’s validity. Nonetheless, this comes with a drawback because building confidence in the validity of the method greatly depends on the case examples taken into account and while the examples given here have proven to be adequate to be used as example problems to validate the method, they were very specific cases, and thus their generalizability is limited. Moreover, the validation was carried out in naturalistic environments, as opposed to controlled scenarios in laboratory environments. While this may be seen as a drawback, we feel that it is in fact an advantage given the suggestions of Design-Based Research (Barab & Squire, 2004). This approach to research of educational artifacts argues that educational innovations (such as the ones produced by Co-CreARGBL) should be tested in a real environment without excluding the context of the educational environment.

It is important to recognize some limitations of this study. On one hand, the method was built in accordance with the participants, thus, biased answers are expected. Future works should work along blind Teachers, using the method only to guide them in order to find out unbiased answers. As acknowledged in the intended uses of the method, the method proposed is not intended as a one-off rapid method for designing ARGBL experiences but to be used in longer processes, which renders it as not useful for teachers just wanting a simple experience for short-term use in a single class. In this sense, perhaps, an authoring tool is recommended. This feature also places some limitations on the validation study because as the validation was based on case studies, those studies needed to span months, thus, the validation case studies were limited to just two (Team A and B). Results on the studies on motivation and learning gains are also limited, as the groups of students were scarce, and the usage of the experiences was only one in a transversal observation. More data is needed to extract generalizable conclusions. Also, due to limitations of the training program’s funding, the observations in the naturalistic environments were limited to three with a qualitative study with few students, so more experiences like these should be enacted to extract further conclusions on the validity of the method.

7 Conclusions and Future Work

This paper has outlined an approach to the Co-Design of Augmented Reality Games for Learning named Co-CreARGBL. It also has shown a validation process conducted with two case studies where teachers and game design and AR professionals built ARGBL games and experiences that were then tested in real classrooms.

The method was applied into two case studies. These case studies were used as example problems in a validation process that used the Validation Square as the framework for validating Co-CreARGBL as a design method.

Arguments have been presented to support the method’s validity. For the validation, a set of Acceptance Conditions were observed, and for each of them, arguments based on the data collected from the case studies are outlined. The validation shows how the method is valid for further experiences and applications.

The results of the validation process suggest that the Co-CreARGBL method is useful in guiding the design, creation, and evaluation of ARGBL artifacts in the classroom. As it has been shown in this paper, the method is meant to guide Co-Design teams made up of Teachers and Designers alike, and it is guided by principles related to Game-Based Learning and Learning with Augmented Reality. With this, it is argued that the original Research Question “What guidelines should Teachers and Designers follow in order to create a motivating and pedagogically effective ARGBL experience?” has been answered as the method has shown the guidelines and the resulting ARGBL games have shown to be effective experiences on learning and motivation by engaging students.

Implications for theory, methodology and pedagogical practice.

Building on the previous arguments and the evidence presented, we argue that this has implications for future research. On one hand, researchers and practitioners are presented with a validated and effective approach on how to design, produce and introduce Augmented Reality Experiences with Games and Board games into the classroom. The complexity of the results is, as the results suggest, explained by the dialogical nature of a method that involves teachers and designers alike. Also, according to the answers by teachers, such a method allows them (i) to be aware of novel and effective technologies such as games and AR that can help them tackle challenges in the classroom while allowing them to innovate (ii) Such a method helps teachers to be more involved on the pedagogical, didactical, and content-wise elements of the experience rather than on the design and development of the actual artifact. (iii) in a similar sense, professional designers while aware and skilled on the practice of designing and developing are rarely aware of pedagogical and contextual aspects.

Results on our design experiments following the Design-Based Research, suggest that this is a useful paradigm when researchers want to explore not only some variables in a controlled context, but when they want to assure that the natural noise of the classroom is involved in the use of the learning artifact. According to this, implication on methodologies for future practitioners include (i) the inclusions of teachers and students early on the design process does benefit the complexity and educational effectiveness of the learning artifact (ii) the iterative nature of design and testing helps designers to improve sequential increments on the artifact being created.

Finally, our results suggest that, as some authors have noted in the past (Hargreaves, 1994; Morales, 2019) educational innovation such as ARGBL should involve important actors such as teachers, school and innovations centers which include collaborative methods to improve the educational practice. The examples presented here and the Co-CreARGBL method are in support of such methods, that adequately adapted may be used to co-create other technological and pedagogical innovations apart from AR and GBL.

This said, future work could include using the method with other ARGBL parameters. For example, other technologies and types of AR and games can be used to discern if there are any differences with the ones used in the cases presented here. Moreover, since Co-CreARGBL proposes a set of considerations to be followed when working on its activities, interesting observations may issue from projects using other considerations, frameworks, or design tools.