1 Introduction

Many students find it difficult to acquire astronomical knowledge. The pupils cannot observe the movements of the planets in the universe, which makes the development of astronomical knowledge abstract and incomprehensible for many students (Cheng & Tsai, 2013). Augmented Reality (AR) technology has been recognized as a promising tool for the transformation of educational practices (Al-Ansi et al., 2023; Garzón, 2021; Pellas et al., 2019). It offers dynamic and immersive learning experiences that engage students in unprecedented ways compared to traditional methods(Phakamach et al., 2022). As the educational landscape continues to change, it is essential to evaluate and comprehend the ramifications of augmented reality (AR) on academic performance, particularly in the context of secondary education (Alkhabra et al., 2023; Pellas et al., 2019).

1.1 Theoretical background and hypotheses development

In recent years, there has been a notable rise in attention to-wards the efficacy of augmented reality (AR) in education (Garzón, 2021; Garzón et al., 2019, 2020). Augmented Reality (AR) supports the learning and academic performance of secondary school students in astronomy in multiple ways. Firstly, it enhances under-standing of abstract concepts by providing a realistic environment where students can visualize and interact with astronomical concepts. For instance, AR can simulate the formation of the Solar System and the methods of detecting exoplanets (Demircioglu et al., 2022; Önal & Önal, 2021). It helps students understand why Earth is unique compared to other planets by showing them the different conditions. Similarly, AR can be used to help students understand the phases of the Moon, eclipses, and tides. They can interact with a 3D model of the Moon to better grasp these concepts (Düzyol et al., 2022). Moreover, this helps them make connections between the real world and virtual objects (Talan et al., 2022).

Secondly, AR increases engagement and motivation by making the learning experience more interactive and engaging. This higher level of engagement leads to increased motivation, which is crucial for effective learning and academic achievement (Faria & Miranda, 2023; Leonardi et al., 2021; Önal & Önal, 2021).

Thirdly, studies have shown that students who learn astronomy with AR technology perform better on achievement tests compared to those who learn through traditional methods (Herfana et al., 2019). This suggests that AR can improve academic performance (Amores-Valencia et al., 2023; Faria & Miranda, 2023; Say & Pan, 2017). Pérez-Lisboa et al. (2020) analyzed the educational intervention with augmented reality and the Stellarium program, in the development of astronomical language, specifically the semantic and morpho-syntactic aspect of the system solar, stars and constellations, for children of five years of age, evidencing an advance in scientific language. However, in the specific context of secondary education and the discipline of astronomy, there is still a requirement for rigorous empirical research to support and augment the existing knowledge base. Most studies have focused on solar system concepts (Önal & Önal, 2021; Say & Pan, 2017; Talan et al., 2022). Therefore, there is a need for studies to assess the effectiveness of AR in other areas, such as light and matter, telescopes, and astronomical observations. There is another gap in the existing studies conducted in secondary education. Most of these studies focus only on the first years, such as 7th grade (Benli Özdemir, 2023; Demircioglu et al., 2022; Say & Pan, 2017). However, there is a need for studies that cover the higher grades of compulsory education as well.

Furthermore, the use of AR in astronomy education can also foster a positive attitude towards the subject. A positive attitude towards learning enhances students’ willingness to engage with the material, ultimately improving their academic performance (Say & Pan, 2017).

Additionally, AR can help students develop visuospatial skills, which are important for understanding astronomical concepts. These skills involve the ability to visualize and manipulate spatial relationships between objects, which is particularly relevant in astronomy (Faria & Miranda, 2023; Zhang et al., 2014).

Another benefit offered by AR is the flexibility to be applied to different teaching multiple disciplicines, especially in Science, Technology, Engineering, and Mathematics (STEM), (Alkhabra et al., 2023; Yen et al., 2013). Today we would have to add another advantage to this list, which is that many of the AR activities can be designed so that students can carry them out at home. If there were to be a temporary suspension of face-to-face teaching again due to a sanitary confinement, the use of this methodology could make work at home more bearable for students (Quicios-García et al., 2020), who could welcome the performance of this type of activity as a complement to some more traditional ones that are proposed in virtual class-rooms.

Lastly, AR applications for astronomy education can be accessed on commonly used devices like smartphones and tablets, making it a flexible and accessible learning tool (Faria & Miranda, 2023; Gallardo et al., 2022).

Although augmented reality shows promising benefits for education, it must also be recognised that it has some limitations. Teachers often lack the necessary training and experience to effectively integrate AR into their teaching. This can hinder the successful implementation of AR in the classroom (Demircioglu et al., 2023). Technical issues during application usage can be frustrating and hinder the learning process, especially for students with disabilities (Quintero et al., 2019). AR applications in education have limited usability due to technical issues that need to be addressed for better effectiveness (Saltan & Arslan, 2017). In a recent review, Aydin and Ozcan, (2022) have found that many of the available augmented reality applications can lead to the same misconceptions described in the literature. While there are benefits of using AR in education, its effectiveness in elementary school education is still limited, indicating a need for further research and development in this area (Madanipour & Cohrssen, 2020).

To maximize the effectiveness of AR, it's crucial to understand that its benefits differ in various learning environments. For instance, AR is particularly beneficial in subjects like science, history, and anatomy, where visualizations and 3D interactions enhance understanding (Gupta & Rohil, 2017). However, its impact may be less significant in language-heavy subjects (Majid & Salam, 2021) that require abstract interpretation. Another example, schools with abundant resources can explore a wide range of AR tools and applications. In contrast, resource-limited settings may find simpler AR experiences (Dick, 2021). Finally, access to technology is crucial for successful AR implementation. Areas with limited technology access face challenges in providing equitable access to devices and reliable internet (Mai & Liu, 2019). This highlights the importance of finding alternative approaches in such situations. Another factor to consider is the characteristics demographic of students. Boys may benefit more from AR than girls. A study found that boys performed better in knowledge post-tests (Salmi et al., 2017). On the other hand, although studies show that AR offers benefits for students of all ages, implementation in early childhood education is limited because the necessary software is not widely available (Piatykop et al., 2022). Curiosity, the AR technology benefits lower achieving pupils the most (Salmi et al., 2017).

The justification for this research emerges from the acknowledgement that traditional teaching methods, while foundational, may not fully exploit the dynamic and evolving ways in which students engage with information in the digital era. Given the increasing prevalence of smartphones and other AR-enabled devices, the educational landscape has the potential to harness these technologies and bring about a transformative shift in classroom experiences. The main objective of this study is to examine the application of Augmented Reality (AR) in the instruction of astronomy during the last four years of secondary education in Spain (levels of 7th to 10th-grade students). We have three research questions:

  1. 1.

    Is there a significant difference in astronomy literacy levels between the pretest and posttest averages of the control group (7th to 10th-grade students) who received traditional teaching methods?

  2. 2.

    Is there a significant difference in astronomy literacy levels between the pretest and posttest averages of the experimental group (7th to 10th-grade students) who were exposed to AR-based instructional interventions?

  3. 3.

    What is the magnitude of the impact of augmented reality on astronomy literacy?

By addressing these questions, we aim to provide insights into the effectiveness of traditional teaching methods versus AR interventions in improving the astronomy literacy levels of students in 7th to 10th grades. The findings of this study can inform educational practices and future research in the integration of technology in secondary education.

In the didactics of the contents related to astronomy, we must highlight the interdisciplinary nature of the teaching of astronomy in secondary education. Physics in charge of elucidating stellar processes and movements in space, chemistry in relation to the composition of celestial bodies, biology in the field of astrobiology, geology contributing to understanding planetary evolution, mathematics describing trigonometry existing in a sundial, and arts and crafts and technology helping to design the necessary measuring instruments for certain observations.

2 Method

2.1 Participants and procedure

We utilised a quasi-experimental design with pre- and posttest assessments and with a control group (Stanley & Campbell, 1973). This type of research “has the identical purpose as experimental studies: to show a causal relationship between two or more variables,” according to (Hedrick et al., 1993). Quasi-experiments, which are analogous to experiments, can evaluate the effects of a treatment or programme when randomisation is not possible, provided that a reasonable foundation for comparison has been established. The distinction between these designs is determined by whether they incorporate an AR application, as determined by an experimental group and a control group.

In this study, we examine three distinct AR applications. This section explains how these applications operate and how they were employed in the study with the participants.

One hundred-thirty (n = 130) students, aged 12 to 16, were chosen from a secondary school in Spain through the use of a non-probabilistic convenience sampling strategy. The individuals participating in this study are currently enrolled in the four final grades of compulsory secondary education (CSE). The study was conducted during the academic year 2021–2022. This research received a favourable report from the University of Salamanca Ethics Committee. Informed consent was obtained from all individual participants included in the study. Furthermore, the parents provided written informed consent.

The details of the students in the sample for each case can be found in Fig. 1. As we can see, in case 1 (7th grade/1ºCSE), there are 17 students in the experimental group and 15 in the control group. In case 2 (8th grade/2ºCSE), there were 21 students in the experimental group and 17 students in the control group. In cases 3 and 4 (9th grade/3ºCSE), there were 16 students in the experimental group and 22 students in the control group. Finally, case 4(10th grade/4ºCSE), there were 8 students in the experimental group and 14 students in the control group.

Fig. 1 
figure 1

The participants by group

Full size image

In Spain, the inclusion of astronomical topics in compulsory secondary education can be found in Classic Culture, Biology and Geology, as well as in Physics and Chemistry (RD 217/2022). (Cahen, 2020) provides a comprehensive examination of the potential opportunities presented by the previous Spanish curriculum for the instruction of astronomy-related concepts.

Several activities have been carried out with augmented reality APPs to address a whole range of astronomical concepts (Fajriani & Masturi, 2023). To carry out the experiences, we will classify the conceptual contents of astronomy in the following categories:

  • Sun-Earth-Moon system: day, night, year, seasons, tides, eclipses.

  • Solar System: planets, satellites, asteroids, comets, meteorites.

  • Stars and Galaxies: physics of the Sun, grouping in clusters, white dwarfs, neutron stars, black holes.

  • Cosmology: origin, dimensions and evolution of the Universe.

  • Position astronomy: observation instruments, constellations.

  • Physics related to astronomy: artificial satellites, rockets, gravitational force.

The experimental and control groups both created identical didactic units for STEM classes. The experimental group utilised the applications StarWalk2®, AR Books (iSolar System and iStorm of Ed. Blume), Star Tracker®, Solar System AR Core®, and Spot the Station® across four cases. The main characteristics of each case are summarised in Table 1. The control group adhered to a traditional approach, utilising textbooks with expository methods and a teacher-centered approach.

Table 1 Summarizes the main characteristics of each case
Full size table

In order to teach the concepts related to those categories, the following AR education resources were used with the experimental groups:

  • StarWalk2 APP: Students began discovering (Fig. 2) the concepts of constellations, zodiac, rocky planet, gas planet, natural and artificial satellites on their tablets. Previous concepts such as the differences between the northern and southern hemisphere were also clarified, and the students were provided with arguments based on the observation of the night sky to refute flat-Earth arguments. The students conducted an investigation into the names of several constellations, such as Orion, Perseus, Heracles, Andromeda, and others.

  • AR Books with an associated APP: The following two books were used; “iSolar System” and “iStorm” of Ed. Blume. Using these books, students discovered concepts like the International Space Station, meteorites, sea tides (Fig. 3).

  • Star Tracker APP: Similar to the Star Walk APP used before, Students played with this APP (Fig. 4) looking for specific constellations, such as the zodiac ones, and other some solar system elements, such as planets and natural satellites on their smartphones.

  • Solar System AR Core APP: With the use of this APP, the students were able to verify the enormous distances that exist between the elements of the solar system, as well as the relative size of the Sun and the different planets.

  • Spot the Station APP: This app is based on NASA's Spot the Station website and provides information that makes it easier to find the space station (Fig. 5). An augmented reality interface makes it easy for users to locate the station and provides options to capture and share images and videos of their sightings in real time. With the help of augmented reality, the compass in the APP shows the students where the international space station is located.

Fig. 2
figure 2

Students using StarWalk2 APP on their tablets

Full size image
Fig. 3
figure 3

Screenshots of two of the AR activities contained in the book

Full size image
Fig. 4
figure 4

Screenshot of one of the constellations finded

Full size image
Fig. 5
figure 5

Screenshot showing the location of the student

Full size image

2.2 Measures

Following Magnusson (1967) advice, we computed item indices typical of Classical Test Theory (CTT). The components of CTT include item difficulty (P), item discrimination (D), and Cronbach’s alpha (α) when an item is removed. P of each item is determined by calculating the percentage of correct answers. According to Allen and Yen, (2001), values between 0.30 and 0.70 provide valuable information for measuring knowledge. Numbers outside of this range are regarded as being fairly easy above 0.7 and severely tough below 0.3. D refers to the extent to which an item can differentiate between individuals who possess more knowledge and those who do not. (Ebel and Frisbie (1991), p. 223) and Price (2017) state that items with D present values above 0.4 are excellent, while those with values between 0.30 and.39 are acceptable but may still need development. Values between 0.20 and 0.29 are marginal, showing that they may need improvement. Lastly, items with values less than 0.19 are poor. They should be rejected or revised. We applied the correlation (rpbs) between the item answer and the scale’s overall score provides as a method for measuring item discrimination (Finch & French, 2019). Every item’s reliability is assessed using Cronbach’s alpha, which should have values around or above a.7 if the item is eliminated. Values around 0.4 for the average variance extracted and loadings above.5 for the indicators are desirable.

2.3 Data analysis

2.3.1 Case 1: Descriptive statistics and item analysis for universe quiz items

According to the results in Table 2, item 6 was the most difficult (P = 0.34), while items 3 was the easiest (P = 0.81). Most items had P values within the recommended range of 0.3–0.7. Based on D, item 2 was the best discriminating item. The scale had a reliability of α = 0.76 and an average variance extracted of 0.42.

Table 2 Case 1: Descriptive statistics and item analysis for universe quiz items
Full size table

2.3.2 Case 2: Descriptive statistics and item analysis for universe quiz items

Based on the results in Table 3, item 2 was the most difficult (P = 0.469) and item 3 and 4 was the easiest (P = 0.64). All items had P values between 0.3 and 0.7. According to D, item 1 was the best discriminant item. The reliability was adequate (α for total scale is 0.75) and the average variance extracted is 0.46.

Table 3 Case 2: Descriptive statistics and item analysis for universe quiz items
Full size table

2.3.3 Case 3: Descriptive statistics and item analysis for universe quiz items

Table 4 indicates that item 7 was the most difficult (P = 0.38), while items 1 to 3 were the easiest (P = 0.69). Using D, we determined that item 6 was the best discriminant item. In terms of reliability, it was satisfactory (α = 0.80) and the average variance extracted was 0.44.

Table 4 Case 3: Descriptive statistics and item analysis for universe quiz items
Full size table

2.3.4 Case 4: Descriptive statistics and item analysis for universe quiz items

Table 5 shows that item 4 was the most difficult (P = 0.37). Items 4 was the most discriminating according to D. The reliability was below 0.7 but sufficient for exploratory studies. The average variance extracted was 0.44, which was appropriate.

Table 5 Case 4: Descriptive statistics and item analysis for universe quiz items
Full size table

3 Results

We first investigated the impact of augmented reality applications (AR APPs) on student learning. We ensured that the control and experimental groups had no significant differences in the pretest. The results indicate that there were no significant differences in the pretest scores between the two groups in all cases. More details are showed in Table 6.

Table 6 Contrast statistic pretest between control and experimental group
Full size table

Once we confirmed the contrast statistic pretest, we conducted a nonparametric Wilcoxon test to evaluate the influence of augmented reality applications. The results are summarized in Table 7.

Table 7 Pretest-posttest contrast statistical differences in control and the experimental group
Full size table

The findings indicate that the control group did not show any significant improvement in most cases, except for case 1 (z: -3.18, < 0.001, MedPretest = 5.60 to MedPosttest = 7.60). The experimental group consistently experienced significant enhancement in all cases (case 1 = z: -3.30, < 0.001; case 1 = z: -2.38, < 0.001; case 3 = z: -3.30, < 0.001; case 4 = z: -1.97, < 0.05) with higher posttest scores (case 1, MedPretest = 5.60 to MedPosttest = 7.60; case 2, MedPretest = 6.00 to MedPosttest = 8.00; case 3, MedPretest = 8.00 to MedPosttest = 9.00;case 4, MedPretest = 4.00 to MedPosttest = 6.00;). There were significant differences between the pre- and posttest data in all cases (case 1, MedPretest—MedPosttest = -1.00; case 2, MedPretest—MedPosttest = -2.00; case 3, MedPretest—MedPosttest = -1.00; case 4, MedPretest—MedPosttest = -1.86). Additionally, according to effect size all these variations are considerated large.

4 Discussion

4.1 Summary of findings

Our study’s findings offer strong evidence that Augmented Reality (AR) is highly effective in enhancing academic performance, especially in secondary education astronomy instruction. The results clearly show the significant advantages of using AR-based applications, as shown by the notable differences between the control and experimental groups in various instances.

Notably, the analysis of the control group reveals that significant improvement was observed only in case 1, where there was a pronounced shift from pretest to posttest (z: -3.18, p < 0.001). The findings of this study suggest that traditional teaching methods may have a limited impact on fostering advancements in understanding certain astronomical concepts. Likewise, French and Burrows (2017) found that the depth of perception of astronomical objects is not accurately learned by students through traditional teaching methods. One reason for this could be that Astronomy deals with numerous abstract concepts (Rattray, 2021). These concepts can be challenging to understand in a traditional classroom environment. Additionally, our findings and of French and Burrows (2017) consistently demonstrate the obstacles faced by conventional pedagogical approaches in effectively catering to the diverse and dynamic learning needs of students, with a particular emphasis on STEM disciplines. The lack of significant improvement in other cases within the control group reinforces the need for alternative pedagogical approaches to elicit meaningful progress in secondary education astronomy.

Conversely, the experimental group consistently exhibited significant enhancements across all cases, demonstrating the robust influence of AR-based instructional interventions. The substantial improvements in case 1 (z: -3.30, p < 0.001), case 2 (z: -2.38, p < 0.001), case 3 (z: -3.30, p < 0.001), and case 4 (z: -1.97, p < 0.05) underscore the versatility of AR applications in fostering a comprehensive understanding of diverse astronomical concepts. Notably, the experimental group’s posttest scores exceeded those of the control group in all cases, providing further evidence of the positive impact of AR on academic performance. The significance of the findings is emphasized by the notable differences between pre- and posttest data in all cases, where improvements were evident despite a relatively high pretest median score.Our results are supported by the work of (Fleck and Simon (2013). Similarly, Faria and Miranda (2023) found through in-depth analysis of 10 peer-reviewed studies that Augmented reality (AR) was observed to yield positive outcomes in students’ learning, motivation, visuospatial skills, and task engagement.

Students today are familiar with electronic devices like smartphones and tablets. This familiarity could be one reason for our results (Faria & Miranda, 2023). This is due to accessibility and flexibility that AR applications for astronomy education can be accessed on commonly used devices, making it a flexible and accessible learning tool (Gallardo et al., 2022). This suggests that augmented reality not only bridges knowledge gaps, but also enables a more profound understanding of complex astronomical principles, even for students who are already familiar with the subject.

Additionally, when examining effect sizes, it becomes evident that all observed variations are classified as large, emphasizing the substantial influence of AR on students' academic achievement. The comparison between pre- and posttest data in our study, accompanied by large effect sizes, resonates with the conclusions drawn by (Benli Özdemir (2023).

4.2 Practices and theoretical implications

The implications of this study are of utmost importance for educational practice, research, and society. The results indicate that incorporating AR applications into instruction can enhance astronomy learning outcomes among secondary students in comparison to conventional methods (Demircioglu et al., 2022). Teachers are encouraged to give thought to the inclusion of AR tools that enable interactive exploration of astronomy concepts in immersive and captivating manners. It is recommended that educational institutions prioritise the allocation of resources for the establishment of necessary technology infrastructure and teacher training programs in order to support the integration of augmented reality applications in STEM subjects (Delello, 2014; Mena et al., 2023).

For research, the present study adds to the growing body of evidence demonstrating the benefits of AR for science learning (Cheng & Tsai, 2013). However, additional efforts are necessary to explore the potential of augmented reality in enhancing classroom instruction, considering diverse subjects, contexts, and age groups. For example, future research could investigate the role of guidance in facilitating AR experiences or compare different AR platform designs (Delello, 2014; Klopfer & Squire, 2008). Additionally, there is a necessity for conducting long-term studies that investigate the effects on motivation and retention of learning (Cao & Yu, 2023).

When considering society as a whole, the inclusion of cutting-edge technologies like AR in education has the ability to increase student involvement in STEM subjects and inspire pursuits in high-demand fields (Ilona-Elefteryja et al., 2020). This is of utmost importance due to the fact that disciplines such as astronomy and other sciences have the potential to inspire curiosity about the natural world, foster advanced problem-solving skills, and cultivate responsible citizenship on a global scale (Fleck & Simon, 2013). The increased utilisation of AR teaching tools may contribute to the solution of the pressing need for a workforce with STEM literacy and a society capable of addressing complex global challenges.

5 Conclusions

In summary, our research makes a significant contribution to the existing literature advocating for the integration of AR technology in education. By building upon and expanding previous findings, our study provides a comprehensive understanding of the transformative potential of AR in various fields. With the rapid advancement of educational technology, the evidence presented in our study, along with previous research, serves as a solid foundation for future investigations and improvements in augmented reality (AR) applications. The collective efforts in this field establish the groundwork for a more engaging, effective, and inclusive educational framework, fostering the development of students equipped with the skills and knowledge necessary in our technologically advanced society.

5.1 Limitations

Although this study offers valuable insights into the advantages of using AR in astronomy education, it is important to acknowledge certain limitations. Firstly, it is important to note that the study was carried out using a sample exclusively from a single secondary school in Spain, thus potentially limiting its generalizability. Conducting the study in various schools and countries would enhance the robustness of the results.

Secondly, the study exclusively analysed immediate learning outcomes through a post-test, without addressing the long-term retention of knowledge or the potential influence on motivation and engagement over time. It is necessary to conduct future research using longitudinal designs.

Thirdly, the selection of AR applications for this educational context was made by the researchers rather than creating custom applications. Incorporating teacher feedback into the application design process could optimise its pedagogical effectiveness.

Fourthly, non-academic factors such as student characteristics, home environment, and quality of instruction were not adequately considered.

Finally, the study did not conduct a qualitative analysis of student experiences and perspectives regarding the utilization of AR tools. The integration of quantitative outcomes and qualitative insights through a mixed methods approach has the potential to yield a more comprehensive understanding.

Although this preliminary study demonstrates promising outcomes, further extensive research is necessary to address limitations and further substantiate the educational efficacy of AR in comparison to alternative technology-enhanced methods. Considering these limitations can facilitate the advancement of the field.