1 Introduction

Emerging technologies could have the potential to transform the education system and are currently considered one of the most engaging ways for students to learn the content of various scientific disciplines (Anđić et al., 2024; Chen & Chu, 2024; Leavy et al., 2023; Moreno-Guerrero et al., 2021). The role of emerging technologies within STEAM education is not fully understood due to the lack of quality theoretical frameworks, practitioner knowledge, and empirical evidence in educational research (Leavy et al., 2023). In particular, little is known about the potential of integrating certain emerging technologies, like virtual simulations (VS), into STEAM learning environments (Thisgaard & Makransky, 2017). A limited number of studies found on this topic dealt with determining the contribution of VS, mainly to the development of the following variables: student achievement, scientific inquiry, reasoning and scientific process skills, interest, goals toward STEM-related careers, STEM awareness, and students’ perceptions of STEM activities (D’Angelo et al., 2014; Sarı et al., 2020; Thisgaard & Makransky, 2017). These studies, as well as the meta-analysis by the authors Perignat and Katz-Buonincontro (2018), which analyzed 44 studies on the topic of identifying the intention of STEAM approaches, singled out the promotion of student engagement as a basic feature of STEAM. Student engagement largely determines all other teaching and learning outcomes. However, this variable was not tested in the mentioned studies but was observed as an indirect construct without determining the method of its measurement. Through indirect observation, it was noticed that the development of student engagement is influenced by instructional practice, the structure of lectures, as well as the interactions of participants in the teaching process (Nicol & Macfarlane-Dick, 2006; Wang et al., 2015). Given the limited amount of research on this topic and certain methodological ambiguities, it remains important that educators communicate their classroom teaching experiences regarding student engagement (Barlow & Brown, 2020). It is particularly recommended to consider factors relevant to the development of this variable (such as instructional practice and content delivery methods) as well as instruments that will directly measure student engagement within STEAM learning environments (Barlow & Brown, 2020). Identified research gaps (1. a limited number of studies as well as a lack of understanding of the role and potential of certain emerging technologies, like VS within STEAM education; 2. a lack of research on this topic within which student engagement was measured directly with an appropriate instrument) served as the basis for the implementation of our research. For these purposes, we selected primary school students and the emotional, behavioral, cognitive, and agentic EBCA scale for assessing student engagement as a 4-component construct developed by the authors Reeve and Tseng (2011). Given that this instrument is intended for students of higher education, we had to modify or adapt it to the needs of our study and check its metric characteristics. With this in mind, we have set a threefold aim. Firstly, we aspired to examine the validity and reliability of the EBCA engagement scale. Secondly, we aimed to examine whether and to what extent the integration of VS in STEAM activities can improve students’ perceived engagement. Thirdly, we strived to examine whether and how the order of integration of VS in STEAM activities affects students’ perceived engagement.

2 Emerging technology integration into STEAM environment

According to the latest reports (Leavy et al., 2023), emerging technologies are defined as tools or software that could have the potential to radically transform the current state of education and thus enable more creative and engaging ways of learning and teaching (Leavy et al., 2023; Sosa et al., 2017). According to this, the following technologies are currently considered “emerging” for education: artificial intelligence (AI), big data, learning analytics, immersive technologies such as virtual reality (VR), augmented reality (AR), and mixed reality (MR), virtual labs and simulations (VS), serious games, robotics, the internet of things, hardware with sensors, wearable devices, and drones (Leavy et al., 2023). Integrating emerging technologies into STEAM activities is considered one of the most engaging ways for students to learn content from various scientific disciplines (Anđić et al., 2022, 2023; Janković et al., 2023; Moreno-Guerrero et al., 2021). The ‘T’ letter in the STEAM acronym is primarily used for solving various engineering challenges, programming, or designing computer graphics. These activities aim to research and design active, innovative, creative solutions and create artifacts (digital or otherwise) by balancing technical expertise with artistic vision and expressing knowledge and skills in the global world (Glancy, 2014; Jones, 2014). Its value, as well as the value of using technology, primarily has long-term aspirations. They are reflected in the transition from a consumer to a producer society, i.e., training the young generations to provide socio-technical contributions using simpler examples in the STEAM classroom and, in the future, complex and useful ones for society (Boy, 2013). Within the STEAM approach, the shift from knowledge consumption to production via emerging technology-enabled tools in a collaborative environment is a way for students to contribute to the closer community, learn from each other, and acquire skills in these areas that are necessary for the future (Boy, 2013).

In a recent meta-analysis, Leavy et al. (2023) reviewed and analyzed 43 qualitative empirical studies on the topic of identifying emerging technologies that are used to strengthen STEAM education. These papers are classified into the following categories: (1) AR/VR/MR; (2) Programming and Robotics; (3) Maker Movement; and (4) Other Technological Applications. The general importance of the integration of the mentioned emerging technologies into the STEAM approach is reflected in the development of 21st -century skills such as creativity, persistence, and problem solving, attitudes towards computing, creative thinking, and learning in this way through promoting engagement, collaborative problem solving, hands-on learning experiences, and providing strong motivation to promote equity (Leavy et al., 2023). However, the authors state that this meta-analysis is limited in providing insight into how emerging technologies can transform and influence learning due to the lack of quality theoretical frameworks, practitioner knowledge, and empirical evidence. Also, bearing in mind that this field is developing, there is a possibility that not all examples of good practice are included due to poor dissemination and recording of the results. Furthermore, we will list a few examples of papers that were not included in this meta-analysis.

In a study by Laut et al. (2015), STEM activities are empowered with robotics to develop and understand the connection between biology and engineering. The students were given the task of completing their biomimetic robotic fish through STEM and letting the finished product be verified at the New York Aquarium to observe the fish’s response in biology. The results of this study showed that robotics can strengthen STEM learning and contribute to students’ understanding of the connection between these two disciplines. In the study by Techakosit and Nilsook (2018), the contribution of the integration of AR within STEM activities to the development of STEM literacy was examined. The results showed that, for learning the STEM contents, imagination, design ability, finding information, and using STEM’s basic abilities to solve problems were very important skills that students develop while learning in this way. Further, Chen and Huang (2020) investigated the contribution of serious game-based learning to the strengthening of STEAM activities from the aspect of improving achievement and reducing the cognitive load when working with primary school students (13–14 years old). The results of this study showed that game-based learning can strengthen STEAM activities and contribute to the development of student achievements and the reduction of cognitive load. Emerging technologies have the potential to initiate inevitable and necessary changes in the educational system through redefining and reshaping teaching that is consistent with STEAM principles (Leavy et al., 2023). What is needed to utilize and maximize the opportunities of these technologies within the STEAM approach is a reconceptualization of school programs, teaching, learning, and assessment methods (Meletiou-Mavrotheris, 2019). We will further look at the potential of integrating certain emerging technologies into the STEAM approach, i.e., we will focus on the integration of VS in the STEAM learning environment.

3 Integration of virtual simulations into STEAM environment

Virtual simulations (VS) imply computer modeling of reality, i.e., computer-based representations of real or hypothesized scientific phenomena and processes, with which students, in an interactive way in a virtual environment, become familiar with the mental models of scientists and construct their own to understand and explain certain scientific phenomena (Falloon, 2019; Sanina et al., 2020; Zhang, 2014). They offer the possibility of observing scientific processes visible and invisible to the naked eye, as well as the possibility of visualizing abstract, less abstract, and non-abstract concepts (e.g., electrons, molecules, light rays) (Maričić et al., 2023; Olympiou et al., 2013). The basic intention of this visualization is reflected in the transformation of abstract phenomena, i.e., theoretical-conceptual constructions into perceptual representations, to build a bridge between the students’ understanding of those concepts in the natural environment and the mechanism of their actual functioning (Sanina et al., 2020). In addition, VS offers the possibility of simplifying the investigated phenomenon or process by highlighting the target elements being observed and removing complexity, or it can be modified to a simpler or shorter time frame to more easily interpret certain natural phenomena (de Jong et al., 2013; Maričić et al., 2023). VS appear in the form of computer-based animations such as models, simulations, and experiments (Falloon, 2019). All these forms offer students the opportunity to enter a micro-virtual world where they can manipulate virtual equipment, materials, and variables of interest and immediately access the obtained results (Scalise et al., 2011; Wen et al., 2020). Through virtual models, simulations, and experiments, students can observe and investigate those natural phenomena and processes that are not easy to observe and investigate in real-life circumstances (Zhang, 2014). In addition to the above, they can be more manageable, more flexible, safer, more profitable, and faster to implement than real hands-on activities (Wen et al., 2020).

When working on VS, students encounter two processes: transformation and regulation. In the process of transformation, students produce direct information by forming hypotheses, designing experiments, and concluding. Through the process of regulation, students connect the variables, conditions, and events presented in the problem, identify key variables, and visualize the conditions of the simulation (Lim, 2004; Sarı et al., 2020). As a result of these processes, we can notice that VS can play an active role in the STEAM learning environment in terms of supporting the research process and providing modeling opportunities (Sarı et al., 2020). These processes can then be carried out through real hands-on activities, which include the integration of other disciplines such as engineering, art, and mathematics. Within them, research support is strengthened through previous manipulation of the phenomenon or process in virtual conditions, visualization of the invisible, simplification of reality, regulation of the time frame, and manipulation of the variable of interest, while the modeling process can be performed more faithfully and creatively through the design of active, innovative, creative solutions using knowledge of engineering and mathematics and the creation of artifacts through balancing real material with artistic vision. Thus, VS can strengthen and support STEAM learning, and students can express their skills and knowledge through different disciplines.

Although VS are considered promising emerging technologies that can support STEAM learning, very little is known about their potential in research practice (Thisgaard & Makransky, 2017). In a meta-analysis by D’Angelo et al. (2014), which dealt with determining the contribution of VS within the STEM approach, 59 studies were reviewed. The results of this meta-analysis showed that VS can strengthen STEM activities in terms of student achievement, scientific inquiry, reasoning skills, and non-cognitive outcomes. Although this meta-analysis showed that VS can strengthen STEM learning, the authors state that it is necessary to carry out more research to gain insight into the benefits of VS within the STEM domain. The research by Thisgaard and Makransky (2017) examined the contribution of VS to students’ knowledge of evolution, interest, and whether simulations could catalyze STEM academic and career development. High school students (18 years old) were supposed to identify an unknown animal found on the beach through VS while investigating various aspects of natural selection and genetics through video displays of genetic links and 3D visualization of a population of a species on an island. The results of this study showed that VS can strengthen STEM learning in terms of developing student interest and goals toward STEM-related careers. Sarı et al. (2020) analyzed the contribution of VS within STEM activities to the development of students’ scientific process skills, STEM awareness, and views on activities. Second-year undergraduate students participated in the research. The results showed that VS can strengthen STEM learning from the perspective of these variables and that students believe that STEM activities provide numerous advantages, such as designing and developing engineering products, conducting experiments, and reducing errors.

The contribution of VS was examined within STEM learning, focusing mainly on the following variables: student achievement, scientific inquiry, reasoning and scientific process skills, interest, goals toward STEM-related careers, STEM awareness, and students’ perceptions of STEM activities (D’Angelo et al., 2014; Sarı et al., 2020; Thisgaard & Makransky, 2017). These studies, as well as the meta-analysis by Perignat and Katz-Buonincontro (2018), which reviewed 44 studies on the topic of STEAM approaches (i.e., identifying the purpose of STEAM education, definitions of STEAM acronyms, and definitions of ‘A’ in STEAM), single out the engagement of students as the basic feature of STEAM education within these disciplines. However, student engagement was not tested in these studies but was observed as an indirect construct without determining the method of its measurement. In the next section, we will focus on this variable.

4 Involvment/engagement theory

The understanding of the concept of student engagement was contributed by Astin (1984), who studied student development for more than 20 years. Instead of the term engagement, Atkin uses the term involvement and focuses on college students. According to him, student involvement refers to the amount of physical and psychological energy that students devote to the academic experience (Astin, 1984). This determination is based on the following five postulates, shown in Fig. 1.

Fig. 1
figure 1

Postulates of student involvment

Full size image

Atkin’s definition was later expanded by the director of the National Survey of Student Engagement, George Kuh, who states that engagement, in addition to the investment of physical and mental energy of the participants in the educational process, also represents the effort of the institution that it invests in using an effective educational performance (Axelson & Flick, 2010). Later, the determinations of student engagement became more and more complex, taking into account different aspects of education, but what they all have in common is that an educational institution with an educational system is not only a place where acquired knowledge is transferred from individual to individual but also a place where different types of relationships develop. These relationships exist between the participants in the educational process (the social component) as well as between the participants and the learning object (the intellectual component), and they are characterized by a certain emotional flow. Bearing that in mind, according to modern understandings, engagement is defined as a state of emotional, social, and intellectual readiness for learning, which is characterized by curiosity, participation, and the drive to learn more (Abla & Fraumeni, 2019). These connections can be observable, like visible behavior, but also unobservable, like internal attitudes. With that in mind, the authors Fredricks et al. (2004) identified three different types of engagement: emotional, behavioral, and cognitive, while Reeve and Tseng (2011) described a fourth type: agentic engagement (see Fig. 2).

Fig. 2
figure 2

Engagement as a 4-component construct

Full size image

Therefore, engagement can be defined as a multi-dimensional construct. Within STEAM education, it has been observed that this variable largely determines all other teaching and learning outcomes for students (Barlow & Brown, 2020; Hong et al., 2020; Khamhaengpol et al., 2021). As previously stated, student engagement in these studies was not directly measured but was observed as an indirect construct. Through indirect observation, it was noticed that the development of student engagement is primarily influenced by instructional practice, the structure of lectures (and exams), as well as the interactions of participants in the teaching process (Nicol & Macfarlane-Dick, 2006; Wang et al., 2015). However, given the limited amount of research on this topic and certain methodological ambiguities within it, it remains important that instructors and educators consider and communicate their practical classroom teaching experiences regarding student engagement (Barlow & Brown, 2020). In doing so, it is particularly recommended to take into account factors important for the development of this variable, including the structure of lectures, content delivery methods, and student interactions, as well as instruments that will measure this variable in a direct way within the STEAM learning environments (Barlow & Brown, 2020).

5 Purpose of the study

Based on a detailed review of the literature, the following research gaps were identified: (1) a limited number of studies as well as a poor understanding of the role and potential of certain emerging technologies like VS within STEAM education; (2) a lack of research on this topic in which student engagement was directly measured with an appropriate instrument. To fulfill the mentioned research gaps, we decided to conduct this study. For these purposes, we selected primary school students and the scale for measuring emotional, cognitive, behavioral, and agentic engagement—the EBCA scale by Reeve and Tseng (2011). As the EBCA scale is intended for high school students, we had to modify it, adapt it to the needs of our research, and check its metric characteristics. With this in mind, we have set a threefold aim. Firstly, we aspired to examine the validity and reliability of the EBCA engagement scale. Secondly, we aimed to examine whether and to what extent the integration of VS in STEAM activities can improve students’ perceived engagement. Thirdly, we strived to examine whether and how the order of integration of VS in STEAM activities affects students’ perceived engagement. The following research questions arise from the stated three-fold aim:

  1. RQ1

    Can the EBCA engagement scale be used validly and reliably in the primary school context?

  2. RQ2

    Whether and to what extent the integration of VS in STEAM activities can improve students’ perceived engagement?

  3. RQ3

    Whether and how the order of integration of VS in STEAM activities affects students’ perceived engagement?

6 Methodology

6.1 Research design

The research was carried out according to the cross-over research design (Crowder & Hand, 2017; Hughes et al., 2022), in which the students of the experimental groups undergo all STEAM (STA and SA) learning conditions but only in a different order. The research design is shown in Fig. 3.

Fig. 3
figure 3

Research design

Full size image

This Pre—Post—Post-Delayed engagement assessment design was used to collect measurement outcomes before, during and after the intervention (Craig et al., 2012). Such a design allowed us to gain insight into whether and to what extent the integration of VS into STEAM activities can improve students’ perceived engagement, as well as how the order of VS integration in STEAM activities affects students’ perceived engagement.

For this research, schools from the district were recruited, and classes of 3rd -grade students that were available to the researcher were selected. A convenience sampling method was applied. The students in the selected classes were given a pre-engagement scale (PES1) to determine the level of their previous perceived engagement in science classes. Those classes of students who showed an approximate perception of previous engagement in the classes were retained in the research. PES1 was used as one of the criteria for equalizing the groups (and as a covariate in the analysis of the results). Selected classes of students were then randomly assigned to one of the STEAM conditions: STA (science, technology, and art) or SA (science and art). Through the combination of these STEAM conditions and the usage of cross-over design, four groups were formed: two control (C1 and C2) and two experimental (E1 and E2): C1 - STA + STA, C2 - SA + SA, E1 - STA + SA, and E2 - SA + STA. After the formation of the groups, the first lesson was held in C1 and E1 (STA lesson), and C2 and E2 (SA lesson). Then the students were given a post-engagement scale (PES2) to determine the level of their perceived engagement after participating in the first part of the intervention. Next week, the second lesson was held in C1 and E2 (STA lesson) and C2 and E1 (SA lesson). After the end of the second lesson, the students were given a delayed post-engagement scale (PES3) to establish their level of perceived engagement after participating in the second part of the intervention.

6.2 Intervention

For the implementation of STEAM activities, the science content Magnetism was selected. The first lesson included the following concepts: what is a magnet, the shapes of a magnet, the poles of a magnet, the lines of force of a magnetic field, attraction and repulsion, and action through different environments. The second lesson included the following concepts: magnetization, magnetic field strength, natural and artificial magnets (make an artificial magnet), and the effect of magnets in different environments (make a boat). These scientific contents are strengthened and integrated with the contents of art: landscape and abstract art (in the first lesson, abstract art, and the second lesson, landscape). In addition to the concepts of abstract art and landscape, elements of visual art are also integrated into the lessons to introduce a science concept. These elements included the following: observing works of art, painting examples of abstract art and landscapes, and creating original works of art that also present scientific concepts about magnetism. Through the integration of the content of the sciences and arts with technology, the STA condition was formed. Technology integration referred to the introduction of VS (from the JavaLab series) on magnetism to strengthen the understanding of the scientific concepts of these contents. VS offers the possibility of visualizing those abstract concepts that students cannot see with the naked eye, such as the lines of force of the magnetic field and their behavior during the approaching of the same and different poles of the magnet, the concept of magnetization, the formation of domains within metals, and their orientation. By integrating the content of the sciences and arts (without technology), the SA condition was formed. Basic STEAM conditions are shown in Fig. 3.

By combining the STA and SA conditions through a cross-over design, two more conditions were formed - STA + SA and SA + STA. STEAM activities will be briefly described below.

6.2.1 STA and SA conditions

All students were introduced to the intervention in the same way. They were told the story of the shepherd Magnus - how the ore magnetite was discovered and how the term magnetism came about. During this conversation, students were shown an example of this ore.

  1. 1.

    STA condition: Lesson 1 – The students were then shown paintings from the series Magnetic North: Imagining Canada in paintings by seven famous Canadian painters (abstract). Through a conversation with the researcher about the paintings, the displayed techniques, and the fascinating name of the entire collection of these works, they came up with the term magnetism. This term is then connected to the term from the story of the shepherd Magnus. Then, through hands-on activities, the students went over the following concepts: what is a magnet, the shapes of a magnet, the poles of a magnet, the lines of force of a magnetic field, attraction and repulsion, and the action of a magnet in different environments. Through VS from the JavaLab series about magnets, students strengthened their knowledge about magnetism, magnetic fields, magnet poles, and magnetic field lines of force. After this part, students were introduced to the concept abstract art. They were shown paintings by famous abstract artists, such as Clyfford Still. The students discussed the paintings and communicated what impressed them, i.e., what was magnetic about them. After that, with the usage of different art materials and media, the students were placed in a position to create their magnetic abstract work. Then, through the main activity, students had to create an original 2D artwork that integrates elements of science and art. The students painted their abstract work of art with magnets through the property of magnets acting through different environments.

    Lesson 2 - The students were shown paintings from the series Magnetic North: Imagining Canada in paintings by Canadian artists, but this time the landscape ones. The researcher introduced the students to the concept of magnetism through a conversation about the paintings, the techniques shown, and the fascinating name of the collection of these works. This concept is connected with the concept from the story of the shepherd Magnus. The students then went through the following concepts through hands-on activities: magnetization, magnetic field strength, natural and artificial magnets (make an artificial magnet), and the effect of magnets in different environments (make a boat). Through VS, students strengthened their knowledge of magnetism and magnetization, magnetic fields, and natural and artificial magnets. After this, students were introduced to the concept of landscape. They were shown paintings by famous landscape painters from the Barbizon School. The students discussed the paintings and communicated what impressed them, i.e., what was magnetic about them. After that, with the usage of different art materials and media, the students were placed in a position to create their own magnetic landscape. Then the main activity was introduced, in which the students had to create an original 3D artwork that integrates elements of science and art. The idea was to create an original 3D interactive landscape—an image of a landscape in which a part of the artwork is integrated, which can be moved by a magnet and make it interactive.

  2. 2.

    SA condition: This condition included the integration of science and art in both lessons, but without technology, i.e. all those elements (in the same order) from the STA condition were represented here, only without the usage of VS.

  3. 3.

    STA + SA condition: Within this condition, the first lesson was performed under the STA condition, while the second was carried out under the SA condition (without technology).

  4. 4.

    SA + STA condition: This condition implied that the first lesson was performed according to the SA condition, while the second was carried out according to the STA condition.

6.3 Sampling

84 3rd -grade students (9–10 years old, M = 9.643, SD = 0.482) from two primary schools in Eastern Europe participated in the research. The classes were recruited from schools attended by students with a diverse body: students from national minorities and different ethnic backgrounds, as well as students who learn according to the IEP. In the research, those classes of students that showed an approximate perception of previous engagement on PES1 and those students within those classes who filled out all three PESs were retained. For this research, four classes of 3rd -grade students were recruited and randomly assigned to one of the STEAM conditions. The random distribution in our research was performed so that already-formed classes were randomly assigned to one of the four STEAM conditions (in each of 21 students). Teacher bias was excluded by introducing a trained researcher into the intervention. Including all students in both STA and SA conditions allowed us to monitor the impact of the order of VS integration on students’ perceived engagement.

6.4 Data collection

Data in this research were collected using a previously created instrument, the EBCA scale, for assessing students’ perceived emotional, behavioral, cognitive, and agentic engagement by Reeve and Tseng (2011). Given that this scale is intended to measure the perceived engagement of high school students, we had to adapt it to the needs of our study to successfully assess the perceived engagement of primary school students. These adaptations were also reflected in the slight modification of the items, which resulted in the creation of three scales: PES1, PES2, and PES3 (for example, on PES1, the items are directed to the state before the implementation of the intervention, on PES2, the items are directed to the state immediately after the first part of the intervention, and on PES3, the items are directed to the state after the implementation of the intervention). Before conducting the research, permission was requested from the author to adapt the scale. Adaptation resulted in several rounds of revision in which some items were excluded. During this process, experts in the field of methodology were consulted, as well as teachers with work experience spanning over 10 years, as the first assessors of the validity of the scale. The revised scale was adapted for 84 primary school students, and for the second round of checking construct validity and reliability, confirmatory factor analysis (CFA) was performed. The scale consists of four blocks, of which the emotional block has four items, the behavioral block has five items, the cognitive block has five items (the original has eight, i.e., three items from this block were excluded), and the agentic block has five items. These items are intended to assess four different types of students’ perceived engagement. Within emotional engagement, the following values were monitored: enjoyment, fun, interest, and curiosity. As part of the behavioral engagement, the following values are followed: careful listening, paying attention, trying hard, careful listening about new topics, and trying hard when starting something new. Within cognitive engagement, the following values were monitored: relating to prior knowledge, relating to personal experience, connecting different ideas into a meaningful whole, creating own examples to understand the concepts, and reviewing what was done. Within agentic engagement, these values are followed: asking questions to make the class active and lively; informing the teacher about personal interests; informing the teacher about the need to improve achievement; informing the teacher about preferences; and suggesting ideas for class improvement. The obtained results for each type of engagement, as well as the discussion about them, will be shown in the next two sections, but in such a way that these data follow each part of our threefold aim.

7 Results

7.1 First part: Construct validity and reliability of the EBCA scale

The skewness and kurtosis values for PES1, PES2 and PES3 are between -2 and + 2 which shows that the data is normally distributed (Byrne, 2010; Hair et al., 2010). Kaiser-Meyer-Olkin (KMO) and Bartlett’s Test of Sphericity tests were used to determine the suitability of the data for confirmatory factor analysis (CFA). The KMO and Bartlett’s Test of Sphericity values for PES1, PES2 and PES3 were found to be statistically significant (p < .000). It was ensured that the sample size was sufficient for data analysis (Tabachnick & Fidell, 2007) (see Table 1).

Table 1 KMO and Bartlett’s test of sphericity values
Full size table

The obtained values were accepted as an indication that CFA could be performed. IBM SPSS AMOS program was used for CFA. In the upcoming paragraphs, we will present the CFA results for each scale.

Within CFA results, we monitored the values of various fit indices, which are primarily used to assess the fit of the model to the data. As a result of the analysis conducted on 19 items, the RMSEA values for PES1, PES2 and PES3 were found to be within acceptable range. Fit indices for PES1 show that this scale fits the overall sample well (χ2 (140, N = 84) = 183.437, p = .008; CFI = 0.977, TLI = 0.972, RMSEA = 0.061, SRMR = 0.076). Covariance of error terms based on modification indices (MI > 20) was created for six pairs which improved the model. The final model is shown in Fig. 4.

Fig. 4
figure 4

STA and SA conditions

Full size image

Convergent validity and composite reliability (CR) of PES1 are also good. All factor loadings have a value above 0.60 (Fig. 4). Average variance extracted (AVE) values are above 0.05, and CR values are above 0.70 for all constructs (Hair et al., 2017). Cronbach alpha (CA) values are also above 0.70. Discriminative validity of the scale is good - the square root of AVE values (bold values) are higher than inter-variable values (below bold values) (Fornell & Larcker, 1981) (Table 2).

Table 2 Validity and reliability of PES1
Full size table

Fit indices for PES2 show that this scale fits the overall sample also well (χ2 (146, N = 84) = 197.611, p = .003, CFI = 0.939, TLI = 0.929, RMSEA = 0.065, SRMR = 0.065) (Fig. 5).

Fig. 5
figure 5

Measurement model of PES1

Full size image

Convergent validity and composite reliability (CR) of PES2 are also good. Factor loadings are above 0.60 (Fig. 5), AVE values are above 0.05, CR and CA values are above 0.70 for all constructs. The discriminative validity of the scale is good - the square root of AVE values is higher than inter-variable values (Table 3).

Table 3 Validity and reliability of PES2
Full size table

Fit indices for PES3 show that this scale fits the overall sample well (χ2 (145, N = 84) = 189.682, p = .007, CFI = 0.948, TLI = 0.939, RMSEA = 0.061, SRMR = 0.064). Covariance of error terms based on modification indices (for one pair - MI > 20) was created for one pair which improved the model. The final model is shown in Fig. 6.

Fig. 6
figure 6

Measurement model of PES2

Full size image

Convergent validity and composite reliability (CR) of PES3 are also good. Factor loadings are above 0.60 (Fig. 6), AVE values are above 0.05, CR and CA values are above 0.70 for all constructs. The discriminative validity of the scale is good - the square root of AVE values is higher than inter-variable values (Table 4).

Table 4 Validity and reliability of PES3
Full size table

Since the data showed a normal distribution, parametric tests were used for further analysis. A repeated measures ANOVA was used to determine the difference in student-perceived engagement between the three different time points. The ANOVA and ANCOVA analysis where used to determine whether there was a difference in the students’ perceived engagement between different STEAM conditions at PES1, PES2 and PES3. An independent t-test was used to determine whether there was a difference in the order of VS integration. These analyzes cover the second and third parts of the aim Fig. 7.

Fig. 7
figure 7

Measurement model of PES3

Full size image

7.2 Second part - contribution of the VS in STEAM activities

One-factor ANOVA analysis of repeated measures compared the difference in students’ perceived engagement between three different time points - PES1, PES2 and PES3. The results of this analysis for all groups indicate a significant influence of time for all types of engagement, i.e. that the level of perceived emotional, behavioral, cognitive and agentic engagement changed significantly during these three-time points (Table 5).

Table 5 The difference in perceived engagement between all three time points
Full size table

These differences were further processed, to establish between which time points within each type of engagement there was a significant difference. These results are shown in (Table 6).

Table 6 The difference in perceived all four types of engagement between all three time points
Full size table

Based on these results, it can be observed that there are significant differences between PES1 and PES2, as well as PES1 and PES3 within each type of engagement, while significant differences between PES2 and PES3 exist within behavioral, cognitive and agentic engagement.

Further analyses considered the differences between the groups. ANOVA analysis found that there was no significant difference in PES1 in terms of perceived engagement (F (3, 80) = 0.484, p = .695, C1 M = 3.386, SD = 0.299; C2 M = 3.429, SD = 0.375; E1 M = 3.470, SD = 0.405; E2 M = 3.512, SD = 0.320). PES1 scores served as a covariate for ANCOVA analysis.

ANCOVA analysis found that there was a significant difference on PES2 regarding perceived engagement (F (3, 79) = 6.980, p = .000, ηp2 = 0.210, covariate under control F (3, 79) = 27.407, p = .000, ηp2 = 0.258). Through further analysis, we tried to determine within which type of engagement and between which groups this difference exists. The results showed that there was a difference in behavioral engagement (F (3, 79) = 3.835, p = .013, ηp2 = 0.127, covariate controlled F (3, 79) = 94.359, p = .000, ηp2 =. 544) between the STA and SA condition (p = .024) and the STA and SA + STA condition (p = .034), where the students of the C1 group (M = 4.324, SD = 0.171) showed significantly better results compared to the students of C2 (M = 4.076, SD = 0.462) and E2 groups (M = 4.124, SD = 0.449).

ANCOVA analysis found that there was a significant difference in PES3 regarding perceived engagement (F (3, 79) = 7.977, p = .000, ηp2 = 0.233, controlled covariate F (3, 79) = 19.732, p = .000, ηp2 = 0.200). Through further analysis, we tried to determine within which type of engagement and between which groups this difference exists. The results showed that there was a difference in terms of behavioral engagement (F (3, 79) = 5.031, p = .003, ηp2 = 0.160, covariate under control F (3, 79) = 82.300, p = .000, ηp2 =. 510) between the STA and SA conditions (p = .003), where the students of the C1 group (M = 4.419, SD = 0.374) showed significantly better results compared to the C2 students (M = 4.124, SD = 0.403).

7.3 Third part - contribution of the VS integration order

The results of independent t-test showed that the p-value is close to the significance threshold t(40) = 1.753, p = .087, but does not exceed it. The students of the E1 group (M = 4.279, SD = 0.170) showed better results compared to the students of the E2 group (M = 4.183, SD = 0.184), which indicates that the VS integration in the first part of STEAM intervention contributes to a greater extent to the development of student perceived engagement compared to integration of VS in the second part of STEAM intervention.

8 Discussion

8.1 First part: Construct validity and reliability of the EBCA scale

The results of our research show that the 4-factor engagement scale model is aligned, i.e., it fits the overall sample well. It should be noted that for the PES1 and PES3 scales, the covariance of the error term was created based on the modification indices for some pairs, which further improved their model fit. In view of this, it is suggested to check the fitness of the model on another sample and, if necessary, make modifications or remove certain items from the scale. Similar results were found in the research of Ritoša et al. (2020), in which the model fit of three constructs of engagement was checked: emotional, behavioral, and cognitive (the construct of emotional disaffection was added to the scale) on a sample of students from preschool (ages 6–7). After the modifications based on the modification indices, this scale showed a good fit. The engagement scale with all four constructs was tested in the research of Maričić et al. (2023) on a sample of primary school students (10–11 years old) and in the research by Zainuddin et al. (2020) on a sample of students from secondary school (16 years old), but regardless of the modifications that were made by the needs of the research, the model fit was not checked. The model fit of the original scale was checked by authors Reeve and Tseng (2011) on a sample of students from high school (over 16 years old). The 4-factor model proved to be adequate. Regarding the convergent validity, reliability, and discriminative validity of the engagement scales (PES1, PES2, and PES3), good results were obtained in our study, and this shows that the engagement scale in this form can be validly and reliably used in an educational context when working with primary school students (ages 9–10). Similar results were observed in the research of Ritoša et al. (2020). It is important to indicate that our results are limited in terms of generalization because the model fit was checked on a smaller sample of students aged 9–10 from Eastern Europe. The modified EBCA scale should be tested in work with students of different grade levels and from different ethnic and cultural backgrounds, which will improve the generalization of the results and affect their applicability on a more global level.

8.2 Second part - contribution of the VS in STEAM activities

The results further show that the level of perceived emotional, behavioral, cognitive, and agentic engagement changes significantly over time, i.e., the longer students are involved in STEAM activities, the better their perceived engagement is. As noticed in previous studies through indirect observation, the STEAM approach can enhance student engagement (Hong et al., 2020; Khamhaengpol et al., 2021). Our study deepens these observations as it provides results generated as a product of direct measurement of this variable. Observed differences are greatest within agentic, then emotional, behavioral, and finally cognitive engagement between all three time points. These observations are consistent with observations from previous studies indicating that agentic engagement offers great potential in terms of enhancing learning (Reeve & Tseng, 2011). Students of all groups perceived this type of engagement the best over time because, during the intervention, an atmosphere was created in which they were free to ask questions, express their opinions, follow their interests, and make suggestions. Agentic engagement is proactive, intentional, and purposeful; it offers opportunities to enrich the learning process by making it more personal, interesting, challenging, and valuable for students; and it develops a constructive contribution to the planning and flow of teaching activities in which students have a say. In order to develop this type of engagement, teachers should provide students with autonomy support, i.e., they need to create classroom conditions in which students feel free to express opinions, pursue interests, and ask questions (Maričić et al., 2023; Reeve & Tseng, 2011). STEAM activities offer that possibility and leave enough space for an optimal level of personalization of the learning process by students, which is very important for improving their perceptions of learning. Our results indicate that the longer the students were engaged in STEAM activities, the more they developed the values of actively asking questions, communicating their interests, the need to improve achievement as well as suggestions for improving learning, the feeling of enjoyment, fun, interest, curiosity, and finally the values of careful listening, focus, and investing effort. In previous research, it was shown that teachers who work with students from higher schools to a significantly greater extent (disproportionately) activate the components of cognitive engagement, while teachers who work with students of lower school age to a significantly greater extent (disproportionately) activate the components of behavioral engagement (Greene et al., 2004; Reeve & Tseng, 2011). The results of our study are not in line with the aforementioned because it was shown that our STEAM activities in students over time activate the components of all four types of engagement so that none of them is disproportionate to the others. Also, when we compare them, we notice that the agentic, behavioral, and emotional components are only slightly more activated over time than the cognitive ones. A similar pattern was observed in the research of Ritoša et al. (2020), where preschool children showed a higher level of emotional, behavioral, and cognitive engagement, but in approximate proportions. This is most likely related to the nature of the STEAM activities and the first student participation in them, where the other three types of engagement slightly prevailed. This problem should be further and more deeply examined in future studies.

In addition to the above, the results of our research show that the integration of VS into STEAM activities over time significantly contributes to the development of students’ perceived engagement compared to STEAM activities without technology (SA condition). Similar results were observed in the meta-analysis by Leavy et al. (2023), in which it was stated that emerging technologies have the potential to increase student engagement, as well as in the study by Katyara et al. (2023), in which it was shown that the integration of different technologies into learning activities can enrich this process and significantly increase different types of student engagement. Over time, in our study, emerging technology primarily encouraged the development of agentic, behavioral, emotional, and finally cognitive engagement. This shows us that the implementation of VS develops the values of personalization, enrichment of content, and learning conditions, then the values of participation in activities, attention to tasks, investment of effort, perseverance, and absence of behavioral problems, and finally the feeling of joy, fun, interest, and curiosity in the students. Kahu et al. (2015) found that positive emotions associated with the topic, such as interest, fun, and enthusiasm, come from learning that is integrated with life experience, as well as the intersection between learning materials and students’ work and experience. STEAM’s technology-enhanced approach offers it all. Considering that the research on this topic is limited, it is recommended to investigate this issue more deeply and further through a longitudinal study, which can provide significant insights into the contribution of emerging technologies to the development of different types of student engagement over a longer period of time. These data would indicate the potential of emerging technologies in maintaining student engagement as well as in the development of different types of student engagement, considering the time frame and acquired experience in STEAM activities.

If we consider the results obtained by comparing all four different conditions and groups (while eliminating the time factor), we can also see that STEAM activities enhanced by VS contribute to the development of student-perceived engagement to a greater extent. These differences are significant in terms of behavioral engagement, where it was shown that the constant integration of VS (through both lessons, STA) within STEAM activities significantly contributes to the development of this type of engagement compared to the STEAM condition without VS integration (SA) and the STEAM condition with partial VS integration (only within the second lesson, SA + STA). A similar observation was made in the research by Garcia-Martinez et al. (2021), in which it was shown that the integration of technology into teaching not only changes the way students learn but also changes their learning behaviors and performance in the long run. Similar results were also observed in the research by Katyara et al. (2023), where it was noticed that the integration of technology in learning activities contributes to the greatest extent to the development of behavioral engagement. These facts are explained from the perspective of various opportunities and benefits that technologies provide to the development of this type of engagement, such as the following: they make the learners more actively involved in the learning process and encourage them to invest more efforts; reduce the dominance of the teacher; enable students to independently participate in more self-regulating learning activities; therefore, help them to develop self-reliance, persistence, and attention (Katyara et al., 2023; Maričić et al., 2023; Zinan & Sai, 2017). This indicates that students who learn content with STEAM-embedded technology tools develop the values of active involvement, attentive listening, persistence, focus, and investing effort to a much greater extent. These facts can be justified by the benefits that VS offer in terms of learning. While the students were learning through them, they were able to visualize abstract concepts - those that they failed to see through real hands-on experiments such as the lines of force of a magnetic field, their behavior during the approach of the same and different poles of a magnet, the concept of magnetization, the formation of domains within metals, and their orientation, which encouraged them to listen carefully, direct their attention, and put in extra effort when working on VS. Thus, students were significantly more actively involved in STEAM learning activities, which had the greatest impact on the development of behavioral engagement. Such results should be discussed in future research from the Technology Acceptance Model (TAM) theory perspective, which would indicate the extent to which students (and teachers) accept this type of technology as well as their future intentions regarding the usage of VS in teaching. In addition to the above, it is suggested that different types of engagement should be correlated with other variables, such as student achievement and motivation, to see their connection and consider other important components of the teaching process.

8.3 Third part - contribution of the VS integration order

Our results also reveal that the integration of VS at the beginning (in the first STEAM lesson) contributes to a greater extent to the development of students’ perceived engagement compared to the integration of VS at the end (within the second STEAM lesson). Similar results were observed in the research of Hughes et al. (2022), which examined the order of arts integration within STEAM activities. The results showed that students who studied life and physical science contents first with the integration of art in STEAM activities showed better results compared to those students who studied those contents in a different order. The order of technology integration can be seen as a significant predictor of student engagement in STEAM activities. Students who first learned with STEAM activities in which VS was integrated showed better results in agentic, behavioral, emotional, and cognitive engagement after the first lesson. These data show us that after the first lesson, the students were significantly more enterprising, behaviorally and effectively active, and invested more mental effort in the learning process, which prepared and encouraged them to continue learning about these contents. Also, the integration of another discipline within STEAM activities at the very beginning of the intervention significantly expands the students’ horizons, which leads to multimodal representation of contents, the generation of new ideas, and a more creative approach to learning (Hughes et al., 2022). Students learned scientific concepts about magnetism through demonstration, performing real hands-on experiments, and creating original works of art (that present scientific concepts), but also through VS, i.e., through different modalities. This leads us to the potential conclusion that the integration of VS within the first STEAM lesson prepared the students for the initial conceptualization and visualization of abstract concepts, which gave them a valid basis and later facilitated the continuation of learning the same content. These activities particularly influenced the development of agentic and behavioral engagement, i.e., they strengthened the student’s optimal personalization and enrichment of the learning process through participation, attention, effort, and persistence. Given that within the groups, approximate mean values were observed in terms of all types of student engagement, we can note that multimodal representation of contents greatly influenced the development of emotional and cognitive engagement as well, i.e., it stimulated the development of a positive emotional state and cognitive functions in students. This has been demonstrated by several STEAM studies, which confirmed that this approach prepares students for learning and reduces cognitive load (expands the working memory space) because abstract concepts become much more accessible through multiple modalities of representation, which also affects the regulation of conceptual inconsistency (Campbell et al., 2018; Maričić et al., 2022a, b; Wahyuningsih et al., 2020). VS offer exactly that possibility - through visualization. Such results should be discussed in future research from the perspective of cognitive load theory, which can shed more light on the contribution of VS to students’ cognitive potentials and their connection with different types of engagement.

9 Conculsion, contribution, implications and limitations

9.1 Conclusions

Based on the analysis of our results, we can conclude that the 4-factor EBCA scale model is aligned and fits the overall sample well, i.e., the engagement scale in this form can be validly and reliably used in an educational context when working with primary school students. STEAM activities can support student-perceived engagement, and the longer students are involved in STEAM activities, the better their perceived engagement is. Over time, this type of learning has the greatest impact on the development of agentic engagement (but not disproportionately compared to other types of engagement). VS emerging technology has the potential to significantly enhance students’ perceived engagement, and the more they work on VS, the more they develop the values of attentive listening, directing attention, and investing effort in learning. When we eliminate the time factor and only compare different STEAM conditions, we can also conclude that STEAM technology-enhanced activities can contribute to the development of student-perceived engagement to a greater extent compared to non-technology ones. This contribution is significant in terms of behavioral engagement, which was achieved through VS integration within STEAM lessons. The order of integration of VS also improves perceived engagement, and students who learn with them first perceive all types of engagement better.

9.2 Contribution

  • Assessment of student engagement in education is of exceptional importance, especially for educators and practitioners, because it has been shown through various observations that it greatly affects all other teaching and learning outcomes of students, and that aspect can improve teaching performance and make it more personal and interesting to them. The modified EBCA scale can be used as a valid and reliable instrument for these purposes in working with primary school students;

  • Based on the assessment of student engagement with the use of the modified EBCA scale, teachers can adjust, dose, and adapt their teaching style, motivational support, and instructional guidance to the needs of students and thereby improve learning. In our study, it was shown that autonomy support, i.e., classroom conditions in which students feel free to express opinions, pursue interests, and ask questions, greatly influences the development of both agentic and all other types of engagement, which has the potential to transform and strengthen learning and bring it closer to students;

  • In addition to the above, the use of this scale in the assessment of student engagement can show teachers how students emotionally, behaviorally, cognitively, and agentically experience teaching activities, i.e., how they react, how they behave, how they learn, and what they undertake within the teaching process, which can direct them and help them in further adequately designing STEAM lessons according to the needs and interests of the children. Our study offers clear insights into this, as well as an example of a STEAM activity that can support teaching practice from this aspect;

  • In previous studies, it was confirmed that teachers who work with students of lower school age focus more on activating the behavioral components of engagement, while teachers who work with higher school students focus more on activating the cognitive components of engagement (Birch & Ladd, 1997; Greene et al., 2004; Reeve & Tseng, 2011), which did not prove to be the best in teaching practice. Assessment of student engagement using the EBCA scale can help teachers focus on redesigning teaching activities, i.e., on balancing and equally activating all types of student engagement, because in this way all components important for the learning process and students themselves can be ensured. The results of our study confirm that.

9.3 Implicationas for future studies and limitations

  • Given that our study is limited in terms of the generalization of the results because the model fit of the engagement scale was checked on a smaller sample of students aged 9–10 from Eastern Europe, the modified EBCA scale could be used for the same purposes in work with students of different primary grade levels and from different ethnic and cultural backgrounds, which will improve the generalization of the results and affect their applicability on a more global level;

  • Within our study, only one variable was tested: student engagement and it is recommended that its number be expanded (for example, variables achievement and motivation could be tested) and correlated with student engagement. In this sense, the modified EBCA scale can be used to assess whether and to what extent different types of engagement can predict student achievement and their motivation to learn. In this way, it is possible to discover which type of engagement predicts to the greatest extent student achievements and motivation, which is essential for teaching practice;

  • Given that in our study only VS was tested within STEAM activities, it is suggested to integrate and test other emerging technologies as well, from the perspective of student engagement. It is also recommended to investigate this issue more deeply through a longitudinal study, which would indicate the potential of emerging technologies in maintaining student engagement as well as in the development of different types of engagement considering the time frame and acquired experience in STEAM activities. Also, it is desirable to connect and discuss the results obtained in those studies from the perspectives of cognitive load theory and TAM theory and address the changes in education that STEAM enhanced with different emerging technologies can bring.