Keywords

1 Introduction

With the release of OpenAI’s ChatGPTFootnote 1 at the end of 2022, artificial intelligence (AI) has become a part of everyday life and social consciousness. AI systems that (seemingly) deliver impressive results are publicly available and easy to use for everyone. Besides the advantages of using such technologies, this also poses potential dangers: Now, even users with little or no knowledge of how the technology works can interact with AI systems. As a result, they might receive the products of AI systems less critically than necessary because the systems are by no means error-free (cf. [16, 18]). The use of such systems is also attractive for students: essays, text translations, and even presentations can be completed almost magically with the help of AI systems (e.g., ChatGPT (See footnote 1) or DeepLFootnote 2). Its generated results, which are presented very convincingly, are often adopted without reflection or further verification, and, in the worst case, false information and explanations are learned. To avoid such problems, all students should acquire basic knowledge about AI systems and how they work to enable them to deal appropriately and maturely with these computer science phenomena [1] in their everyday lives. However, the speed with which innovative topics such as AI are integrated into school curricula, teacher training, and teaching materials often does not match the pace of technological progress. Consequently, in many cases, critical future competencies are only established in schools with a delay of several years. Nevertheless, in the meantime, the topic of AI can already be found in several curricula as it was included in recent reforms (e.g., grammar schools in BavariaFootnote 3, AI4K12Footnote 4). There are numerous initiatives and proposals to implement AI in secondary schools [12, 14]. However, we are a long way from the goal of every student acquiring essential competencies in AI at school. So, we designed a portable learning lab on artificial intelligence to address this problem and close the development gap between technological progress and school. To be independent of school type, previous knowledge, age, or other specifics – we want to make AI concepts accessible for every student–, our learning lab uses 25 learning activities, both digital and unplugged, to teach fundamentals, applications, and research topics in AI. In this way, teachers are supported in introducing the topic of AI, which also brings new professional and educational challenges for the teachers themselves. The following sections describe the conception and structure of such a learning lab, illustrate example activities, and present the results of a pilot study on the lab’s perception and assessment.

2 Research About Learning Laboratories

The term learning lab or student lab describes a broad range of out-of-school learning opportunities offered by different institutions, primarily in the STEM field. According to Haupt [3], these are permanent, out-of-school, or extra-curricular learning establishments that use special equipment for STEM subjects. The learning lab provides access to innovative and exciting topics beyond the curriculum and links them to students’ personal experiences. In doing so, students are challenged to explore and act independently; the level of task difficulty is adapted to the target group. Classic learning labs are characterized by references to the curriculum and are visited together on a field trip. In contrast, portable learning labs visit schools with their equipment and organize regular activities under professional supervision ([3], https://www.schuelerlabor-atlas.de/kategorien).

According to Priemer et al. [15], the goals of learning labs include communicating the social significance of scientific content, reducing fears, and attracting students to the STEM field (cf. also [3]). In addition, lab goals can also be teacher-related by integrating them into teacher training and further education and passing on suggestions for lesson planning by using them. Furthermore, learning labs are frequently used as “laboratory schools” for educational research or places of science communication for companies and research institutions.

The influence and effectiveness of learning labs have been investigated in numerous studies, especially in the natural sciences, focusing on motivational factors and interest. A heterogeneous picture of the effectiveness of learning labs emerges. Concerning interest components, Priemer et al. [15] subsume in a meta-study that learning labs can increase students’ interest initially but that these effects are short-lived. It is emphasized that a detached learning lab cannot achieve sustainable effects. Nevertheless, intensive preparation and follow-up of the visit must occur in the classroom context, in which Glowinski et al. [2], among others, observe a more durable interest. Itzek et al. [4] also emphasize that although visiting a learning lab contributes to a higher practical competence of the students compared to regular school lessons, it does not lead to a theoretical understanding of the contents and methods. Instead, it is necessary to integrate the learning lab visit into further teaching. This is also connected to the open and unstructured nature of the work in the learning lab compared to formal school lessons, in which the students achieve the best learning performance. Leiss [7] investigates the influence of learning labs on students’ ideas in the field of physics and finds that learning labs can help to give students an up-to-date picture of research in the natural sciences and influence their ideas in this field.

3 Developing a Learning Lab About Artificial Intelligence

Existing computer science learning labs offer different activities on AI. For example, the Infosphere Laboratory at RWTH Aachen University includes a module on reinforcement learning for upper secondary school. The AI teaching-learning lab projects addressed by Lensing [8] are laboratories used in the context of higher education in engineering courses to link theory and practice. They have a very high degree of immersion as they aim to train subject experts. The laboratory presented in the following is dedicated to the subject area of AI, enables integration into lessons, and is set up directly at schools. Due to these characteristics, the laboratory is not an out-of-school establishment, as is typical for classic learning laboratories. Instead, it also aims to integrate innovative methods into learning at school and qualifies as a portable lab.

The designed learning lab pursues different objectives: Since previous research results (not only among students) indicate a very low level of knowledge about AI (e.g., [10]), basic AI-related competencies are addressed. The project wants to impart knowledge about AI, arouse interest and fascination for the technology and its future potential, and, at the same time, convey the urgent need for action in the field of AI in educational, political, and ethical terms. The students gain competencies for interacting with AI, but also for their future careers. The lab focuses on fundamental questions about AI such as: Where do AI systems already affect everyday life? What can AI systems do in practical and theoretical terms? What should we be afraid of? In addition to central concepts of AI (e.g., narrow and general AI, supervised learning, etc.), research areas, industrial and everyday applications, and social issues are equally included. For this purpose, experts from the respective disciplines participate in the conception of the learning lab, permitting different perspectives on the topic.

The learning lab also enables a holistic approach to the innovative topic of AI in the school context, independent of its curricular integration. The activities are designed to meet the requirements of different types of schools and different grades without being confined to a specific curriculum. This is realized through various accompanying materials and the possibility of differentiation within the scope of the activities. The learning arrangements of the lab, which always contain an information text in addition to the actual task and do not require any guided instruction, enable the students to explore the phenomena, questions, and applications of AI on their own and experimentally and playfully. During this experimental work, the students implicitly deal with the underlying concepts made concrete through the enclosed information texts.

The learning lab consists of 25 learning arrangements, each of which is integrated into an easily transportable wooden box of 60\(\,\times \,\)60 cm. These can be attached to school desks so that an average classroom is sufficient. The boxes include unplugged tasks, games, and digital and technical elements. The materials developed for the learning lab, as well as the conceptual drawings of the boxes, are made available free of charge (https://www.kiki-labor.fau.de/) to enable teachers to replicate the boxes as well as to use the materials in class and to link lessons to (precedent) lab visits.

Besides the differentiation possibilities in content, the lab also shows variability in use. First, the boxes are placed randomly in the classroom and have no fixed order. Second, the participants can choose in which order they want to visit the boxes. However, teachers can provide a specific task for the visit, select which boxes are used, or set a predefined course through the lab. This is supported by the information texts and additional materials in each box, which provide information about thematically related boxes. In general, tasks are helpful to both motivate the reading of the accompanying texts and consolidate the concepts acquired in the lab or to prepare for the following lessons. Finally, the number of students interacting with a specific box can be varied (e.g., based on the students’ age or task). Most boxes can be used by a single student or in groups of two or more students. However, some boxes are explicitly constructed as “multi-player” boxes and require at least two students.

Following the lab’s aims, its limitation to 25 boxes, and a division into basic (Box 1–11) and application and research-related activities (Box 12–25), it is necessary to limit the included concepts to central, general educational aspects, which secondary school students can acquire self-directedly. To determine suitable topics and concepts, suggestions were systematically drawn from the literature [9, 11, 13, 17]. Table 1 outlines the central aspects taken as a basis for the subsequent development of prototypes for student-activating tasks. The concepts were coded at a rather general level (similar to the presentation in the respective papers) to correspond to the level of concepts aimed at in the laboratory activities. For the design of the activities and the corresponding texts, descriptions of competencies in the literature were attributed to the (knowledge) concepts they are based on. The current box-concept assignment is incomplete, as not all boxes have been fully planned and built yet.

Table 1. Collection of AI concepts and competencies underlying the lab
Full size table

Interested students tested these tasks in an informal setting (workshop, out-of-school, or in class): The students were observed by researchers while carrying out the activities, and difficulties, problems, questions, insights, and reactions to the tasks were noted. In addition, the students were asked how they liked the tasks, which aspects were unclear, which improvements and changes they would suggest, and what they had learned or thought they were supposed to experience. Based on these findings, the tasks were modified, refined, and retested. The process is exemplary illustrated with the activity Reality Taboo (Box 9):

Underlying Concepts to be Taught: Reality is extremely complex and abstract. Processing and representing this complexity is difficult or impossible for machines or AI systems.

Activity Prototype: The students work in pairs; one student receives a picture and describes it to his/her partner. The partner makes a drawing based on the description. Certain words are “taboo” when describing, e.g., horse, legs, head, ears, tail for a horse picture.

Observations of the 1st Student Test:. Paraphrases are very easy to find. Moreover, complex, abstract terms can still be used. Therefore, it is not clear that machines do not rely on abstract concepts at all.

1st Revision: Instead of not being allowed to use certain terms, students are limited to the use of certain types of terms: Geometric shapes (examples are given), colors, types of lines, directions, and positions.

Observations of the 2nd Student Test: Examples of geometric figures confuse students if they do not know them (e.g., ellipse). In addition, the students should not choose the picture to be described themselves but always use the uppermost picture card. Otherwise, they only select easy motifs. A supervisor role can be introduced to control adherence to the term limitations if necessary. Concepts are now understood.

Final Revision: Minor conceptual adjustments to students’ vocabulary and knowledge (geometric figures) and selection of final images for the box.

Following the prototype tests, a concept was developed to implement the respective activity in a wooden box. This involved working with a carpenter to create high-quality, stable, long-lasting materials. As part of the final design and manufacturing of the boxes, the materials were also professionally designed.

4 AI in a Box: Exemplary Description of Activities

The portable lab includes boxes that present applications of AI systems or current AI research and boxes that explain general functional/technical principles of AI. Furthermore, both unplugged and digital activities are used in the lab. Three boxes that represent these different types are presented in the following, the other activities are described on the website.

Fig. 1.
figure 1

“Oracle-Cops”

Full size image

4.1 Application-Related Unplugged Box: “Oracle-Cops” (Box 23)

Inspired by the predictive policing AI software developed by the US company GeoliticaFootnote 5 and used by US police forces, this box aims to illustrate how AI systems can be used to predict and prevent crimes. Students work on this activity in pairs or small groups. The box includes a large city map with labeled streets, activity cards, and blue and yellow tokens. Every activity card (cf. Appendix, Fig. 4 for examples) describes a property or incident related to a particular street on the map. The students’ task is to discuss and decide whether the described incident or property can cause a crime in the respective neighborhood. If they assume this is the case, the place is marked with a blue token, otherwise with a yellow token. No token is placed if students expect no positive or negative effects of the described circumstance. After evaluating all events, the students decide, based on the ratio of blue and yellow tokens for each street, whether a police patrol should regularly visit it or not.

By performing this task, students take over the function of the AI system, which, based on past crime data and other socio-economic information about specific neighborhoods, also makes recommendations about preferable routes for police patrols. Their discussion of the “perfect” route and their evaluation of the incidents is essential to understanding that their subjective ideas and opinions are part of their decision and assessment. This also applies to AI systems when trained with data collected and generated by humans. After finishing the activity, the students read a text that explains how the Geolitica system works, how such algorithms can misjudge the conditions in specific neighborhoods, and which factors might contribute to the development of crime according to science.

The box represents an example application of supervised learning already used in everyday life and aims to convey the following ideas: Data stored in companies and government agencies can be used to train AI systems. With them, the systems learn to predict certain aspects, such as the probability of crime by concluding certain events from specific factors. This is not done objectively but is subject to human bias, as this bias is inherent in the underlying data and might also be caused by humans collecting and processing the data.

In a follow-up discussion in class, the activity can be used to illustrate AI applications in society and to discuss ethical aspects of AI: Which problems can arise from AI using data that is biased, i.e. includes stereotypes and prejudices? What might happen in disadvantaged areas when they become a police focus due to AI algorithms? Other application-oriented boxes in the lab that have already been completed present an approach to using AI systems for the control and early detection of epidemics, or take a look behind the scenes of the Spotify algorithm and illustrate how AI systems can be used in medicine.

Fig. 2.
figure 2

“Wanted: AI”

Full size image

4.2 AI Principle Unplugged Box: “Wanted: AI” (Box 1)

This box allows students to get to know phenomena from the field of AI and to delimit them from “normal” computer science applications. For this purpose, the box provides the students with wooden plates that depict everyday objects and applications (examples depicted in the Appendix, Fig. 5). In partner work, the students discuss whether the pictures show AI systems and arrange them accordingly in the box. The plates depict clear examples, such as a mixer, a printer, or a digital assistant, but also examples that encourage further discussions, such as spam filters or the control of power grids. On the one hand, the activity aims to show students’ existing encounters with AI systems in their everyday lives and provide them with orientation points concerning the topic. On the other hand, misconceptions about using AI algorithms in certain products that students may have are unmasked. This is particularly important to help students develop an appropriate mental model of AI technology.

After arranging all example plates, the students can check their results with a UV flashlight as the recommended attribution is attached to the plate with a UV marker. In the following, the informational text describes why specific applications are AI systems or not and briefly explains how they work. Furthermore, it is highlighted that AI systems use particular kinds of algorithms (currently) created for one unique application and do not possess human skills like thinking. The activity examples can also be used for follow-up discussions in class: A definition of AI can be developed based on the applications, concrete AI methods can be explored based on the examples seen, and limits of AI applications in everyday life can be discussed or even tested with the objects and applications.

Besides Box 1 and the Reality Taboo presented in the preceding section, the lab includes several other unplugged boxes on AI principles and different AI paradigms. For example, students do a matching activity on the history of AI, create decision trees by using training and test data in a supervised learning setting, or train a robot to draw simple figures using reinforcement learning.

Fig. 3.
figure 3

“Artist Unknown”

Full size image

4.3 AI Principle Digital Box: “Artist Unknown” (Box 10)

Students working with this box participate in a quiz that presents them with art pieces (photographs, artworks, texts, music, videos) and asks them to assess whether the artist is a human or an AI system. The box consists of a touch screen on which the individual artworks are displayed and where students can vote. Music and sound files are available via headphones. After making their choice, the actual solution is revealed to the students, and they can take a closer look at the artwork again. Each quiz game consists of six art pieces, and several rounds can be played because new items are chosen randomly from an extensive collection of media in each round.

This box aims to show students that AI algorithms can learn to generate creative artifacts. However, AI systems are not inherently creative but either use patterns that the algorithms have identified in human-created artworks to create new media or determine through trial and error what, for example, realistic photos look like. The corresponding informational text illustrates how AI systems can generate pictures using Generative Adversarial Networks (GAN) and shows, which details can help differentiate between authentic and generated images, as it can often be challenging to identify digital media and art as AI-generated. Being able to assess and recognize the results generated by AI systems and to critically reflect on digital media represents an essential skill for the students in their daily lives as fake news and media are getting more common and are becoming harder and harder to identify. The achievements of AI systems, as well as the limits of their creations, can also be a topic of the lessons following the lab visit, especially since AI artifacts can be created easily and (mostly) free of charge on various websitesFootnote 6. Other boxes of the lab that use digital media permit creating deep fakes, exploring face recognition software, or observing how the output of AI systems varies based on the input data used for training. This includes the deliberate use of biased data sets that represent critical prejudices.

5 Evaluation

In the first evaluation, we wanted to know if the learning lab is perceived differently by students with a high and low affinity towards the subject of computer science (CS). Consequently, students’ interest in CS was surveyed, as well as their computer science aptitude self-concept (pre-test survey). Following the lab visit, the students’ self-concept concerning the learning lab itself was gathered.

A second goal of the evaluation was to find assessment methods that can be used permanently in the day-to-day use of the learning lab to enable a constant, easy-to-interpret evaluation of cognitive and non-cognitive facets of the students without disturbing the learning lab experience with questionnaires. For this reason, the assessment of students’ interest was realized with a teacher questionnaire (questions on the students’ CS skills and interests) on the one hand and a voting box survey for the students on the other hand. For this purpose, students were asked to throw a token (labeled with a number for each student to be able to link different surveys while securing anonymity) into one of five voting boxes corresponding to their CS interest (on a five-point Likert scale from “not at all interested” to “very interested”). By surveying the same aspect twice, the concordance between teacher and student perception was to be determined.

To answer the third question guiding the evaluation, namely to investigate whether fundamental concepts of AI are understood and remembered after being confronted with them in the lab, another non-questionnaire method was used: a digital quiz (KahootFootnote 7) to test the students’ conceptual knowledge after visiting the learning lab was piloted. The instrument included seven items in a single-choice form. A sample question looks like this:

AI-Systems...

        A: are always error-free due to the unambiguity of calculations

        B: know your personality and character based on your data

        C: collect data about you to make accurate forecasts

        D: are the more erroneous, the more data they get (multi-tasking error)

Concrete learning processes in the context of working on the boxes were not yet part of the evaluation and represent a starting point for future work. Aspects of students’ behavior and interaction with the boxes were also examined during the workshops but cannot be presented here due to space limitations.

The self-concept evaluations were conducted with regular closed questionnaire surveys. Similar to the voting box, the data of the questionnaires were linked with numbers distributed to the students. Before visiting the lab, students answered four-point Likert-scaled items about their computer science aptitude self-concept based on Köller’s scale [6]. After working with the lab activities, their self-concept concerning the learning lab was surveyed with four items (four-point Likert-scale, adapted from Kauper [5, p. 27]). Furthermore, we evaluated the students’ motivation (12 items, Short Scale Intrinsic Motivation (KIM) [19]) to rule out a (negative) influence of motivation on the results of the self-concept studies. Since the survey took place in a German-speaking country, established instruments already available in German were used.

5.1 Results and Discussion

In 90-minute workshops, ten learning activities were used with students of a grammar school’s 10th and 11th grades. 53 10th-grade students (13 female, 25 male, 2 diverse, 13 no information) and 18 11th-grade students (5 female, 11 male, 2 no information) participated in the survey. While cleaning the questionnaire data, eight incomplete data sets were excluded, resulting in the final evaluation of 63 data sets.

The teachers assessed the CS skills of all the participants as average. The students’ interest in CS was rated as high in grade 11, corresponding to students’ self-assessment in the voting box (M 3.66, SD 0.44). In the 10th grade, one class is rated as interested in CS, while the other is somewhat not. Here, students’ assessment is contrary to the teachers’: Class 1 (M 2.72, SD 0.74) rates its interest lower than Class 2 (M 3.3, SD 0.53), and both classes show some interest in CS. Therefore, the teacher’s assessment of interest can not replace the survey of the students, since the perspectives deviate. This may be attributed to the respective teacher’s concrete implementation of the CS lessons. Concerning the evaluation form, the voting box should be preferred over the teacher questionnaire as it allows individual instead of a global assessment of all students.

The self-concept (SC) items related to computer science aptitude [6] and the learning laboratory [5] were considered in the context of factor analysis; all items show medium to high loadings on the corresponding factor and can therefore be combined into one characteristic value each. On average, the students’ CS aptitude self-concept is good (M 2.00, SD 0.67). The learning laboratory self-concept with a mean value of 3.08 and a standard deviation of 0.56 also shows a positive tendency. All motivational factors (interest/enjoyment, perceived competence, perceived choice, pressure/tension) are evaluated positively based on the mean values. The motivational factors correlate with the learning lab SC of the students at a weak to medium level (r\(_{Interest}\) = 0.41, r\(_{Competence}\) = 0.62, r\(_{Choice}\) = 0.25, r\(_{Pressure}\) = −0.18). Thus, it can be seen that a high motivation of the students during the task is essential to achieve positive effects on the learning laboratory-related self-concept.

In the context of the learning lab, a particular focus is on whether this instruction can also reach students whose aptitude score in CS is relatively low. So, the results were divided into four groups: students with high or low computer science aptitude \((M < 2.5/>=2.5)\) and high or low learning lab SC \((M > 2.5/<=2.5)\). The results showed that seven of the nine students who rated their CS aptitude as low had a positive self-concept about the learning lab and got along well. Thus, the learning lab seems suitable for inspiring students who rate their basic computer skills as low. However, ten out of 54 students with a positive CS attitude evaluate their work experience with the learning lab negatively. The cause of this cannot be answered with the available data: This tendency is not dependent on gender, group, or CS interest. This group possibly evaluates the learning lab as not computer science-oriented enough due to its focus on unplugged or playful elements. However, this cannot be confirmed without further investigations.

Using the Kahoot quiz, the students show their knowledge without a test or survey character, and the short-term competition situation motivates them. A quiz of this kind can thus be integrated as a permanent component of the learning laboratory. However, testing conceptual knowledge presents a challenge regarding content: The piloted items currently do not have the required discriminatory power and therefore need to be revised regarding wording and distractors. In addition, the items have very different levels of difficulty.

In the context of these results, no meaningful correlations between self-concept and conceptual knowledge can be established. Thus, it is also impossible to conclusively state whether the intended concepts are understood in the learning laboratory. However, the overall high number of correctly answered questions is a positive indication. In addition, the informal observations during the school tests of the box design also indicate that working through the learning activities facilitates the understanding of central concepts of AI.

6 Conclusion

The pilot study shows indications that the learning lab can also reach students with a low aptitude self-concept and make CS topics accessible. The influence of the learning lab on the students’ conceptual knowledge of AI cannot be assessed at the moment due to the low significance of the results of the knowledge items. Still, it must be investigated in more depth in further studies. In the context of additional surveys, an iterative improvement of the existing items by adapting the wording and revising the distractors is therefore intended, as well as the addition of further items for the other boxes, to finally clarify whether fundamental concepts of AI can be taught with the help of the learning lab.