Introduction

The science of cosmology has over millennia aimed to piece together a coherent evidence-based story that has energised people from the earliest times—the story of the Universe.

This story has been constructed by finding answers to some of the most fundamental questions and mysteries in the Universe: What is the size of the Universe? How old is the Universe? Where are we located in the Universe? Are there other Universes? Are the laws of Physics the same everywhere in the Cosmos? There is often the perception that cosmology only started in the twentieth century; however, cosmology is perhaps the oldest science (North, 2008). Various cultures around the world have made observations and models of our universe to within the limits of observation. This has allowed them to construct a rich story of the Universe through their own knowledge systems and ask similar ‘big’ questions (Clarke, 2015; Hamacher, 2022; Jegede & Okebukola, 1991; North, 2008; Penprase, 2011; Venkatesan & Burgasser, 2017; Venkatesan et al., 2019). Therefore, it is unsurprising that cosmology excites the imagination of both the public and scientists and connects us across cultures. Furthermore, it provides a rich landscape for discussions in the science classroom. Cosmology as a science explores the large-scale structure of the Universe and the fundamental mechanisms shaping the Universe we observe. Understanding this structure requires looking at its many components which include galaxies, dark matter, dark energy and much more. Cosmology, owing to the stupendous distances involved, looks back at the Universe’s past and aims to predict its evolution into the distant future. It should be emphasised that astronomy, astrophysics and cosmology are symbiotic fields, with distinctions owing to the focus of the work. For example, astronomers who study type Ia supernovae (a particular class of exploding stars) in distant galaxies to calibrate a distance standard are in essence doing cosmology, using the techniques from observational astronomy and astrophysics. From a human perspective, cosmology challenges our place in the Universe.

Conceptions, Misconceptions, Alternative Conceptions

Over the years, research has used various labels for students’ ideas such as preconceptions, misconceptions, alternate conceptions and alternative frameworks (e.g. Driver & Easley, 1978), each implying a different status for these ideas. Earlier studies characterised ‘misconceptions’ as errors needing to be recognised and eradicated, but increasingly the term ‘alternative conceptions’ or ‘frameworks’ have recognised them as viable but non-scientific ways of interpreting phenomena, often useful bases on which scientific concepts can be built (Smith et al., 1994; Tytler, 1998a, 1998b). Further, researchers became interested in characterising progressions in students’ conceptions as they moved towards scientific understandings, investigating children’s developing ideas in mental models of the earth (Vosniadou & Brewer, 1994) or about animals (Carey, 1985). For older students, particularly in Physics, there has been considerable research around the construction of ‘concept inventories’ that characterise progression in key conceptual areas such as force (Hestenes et al., 1992). Such characterisations of progression in conceptions are very useful for informing teaching and learning, and assessment, in these conceptual domains. The current study investigates students’ conceptions in cosmology with the aim of identifying a progression in reasoning that will inform the development of a concept inventory for cosmology. This concept inventory has subsequently been developed and validated (Salimpour et al., 2022). The current paper describes the establishment of the reasoning framework that helped make sense of students’ cosmology conceptions, as preparatory to that inventory.

This research sits within a wider program of investigation of student engagement with authentic, contemporary STEM practices through data analysis and visualisation of real-time telescope data (Fitzgerald et al., 2016; Salimpour et al., 2021a, 2021b). Such an approach is consistent with current thinking on STEM education as diverse, but involving authentic problem solving, contemporary settings, creative design work (Anderson & Li, 2020; MacDonald et al., 2020; Roehrig et al., 2021) or representation construction (Tytler et al., 2013) and visualisation practices (Salimpour et al., 2021a, 2021b). The contemporary astronomy research field is fundamentally interdisciplinary, involving STEM disciplines (e.g. physics, mathematics, computer science) and creative modelling using arts-related practices (Salimpour, 2021, Salimpour et al., 2021b; Salimpour, et al., 2020a). As with any transdisciplinary settings in STEM education, attending to disciplinary depth and coherence in students’ conceptual engagement is critically important (Tytler, 2020). The current paper represents a program of identifying core disciplinary conceptions in cosmology relevant to this wider program.

Previous Research on Cosmology Conceptions

One key characteristic of astronomy topics in general, and cosmology in particular, is the vast scales and distances involved. Students require an array of reasoning tools to grasp the relative scales and be able to think in orders of magnitude. There have been various studies which have highlighted the conceptions held by students with regards to distances in astronomy and the challenges they face with conceptualising those distances (e.g. Coble et al., 2013; Gingrich et al., 2015; Lelliott, 2010; Miller & Brewer, 2010; Rajpaul et al., 2018). Furthermore, astronomical concepts require the student to perceive objects and motions in 3D space through coordinating Earth and space reference frames, often involving immense temporal scales (e.g. Bobrowsky, 2005; Brock et al., 2018; Eriksson et al., 2014). This is a further complication in cosmology because the vast distances also have a temporal component given the finite speed of light—lookback time.

There is a body of research the explores student alternative conceptions in astronomy; these include, for example, review studies (e.g. Bailey et al., 2004); general astronomy topics (e.g. Sadler, 1992; Schneps & Sadler 1987); the seasons, day/night cycle, lunar phases, eclipses (e.g. Danaia & McKinnon, 2007; Dankenbring & Capobianco, 2016; Lelliott, 2010; Slater et al., 2018; Vosniadou & Brewer, 1994); Earth in space (e.g. Liu, 2005); stars (e.g. Agan, 2004; Bailey et al., 2009, 2012); and the shape of the Earth and gravity (e.g. Sneider & Ohadi, 1998; Treagust & Smith, 1989; Vosniadou & Brewer, 1992). Within a single undergraduate course in astronomy, Comins (2001) identified over 500 unconventional conceptions.

From our experience in working with students and teachers in various countries, we see that alternative conceptions tend to be consistent among individuals across the world, grounded in everyday experience and the everyday language that is powerful in framing our thinking. For example, if a person stands outside and observes the sun, moon and stars rising and setting, the first and most intuitive deduction is that Earth is at the centre, and everything moves around the Earth—the geocentric model. The heliocentric model is at first counter-intuitive, because if we are moving, then why do we not feel this motion? Common language often presumes a geocentric perspective. When we speak of sunrise and sunset, we are in fact, already imposing a geocentric frame. Several conceptions hinge around the difficulty of coordinating spatial perspectives, including scale, reference frames, presumptions of Euclidean space and the removal of an absolute directional grid, coupled with everyday spatial presumptions based on language and experience.

Although there is a vast body of work focussing on student conceptions in astronomy, there is limited work focussing on cosmology at high school. Various studies, mostly at an undergraduate level, with a small number at high school level have highlighted the variety of conceptions that students have with regards to topics in cosmology. These conceptions relate to the expansion, evolution and fate of the Universe (Conlon et al., 2017; Lightman & Miller, 1989; Prather et al., 2002; Wallace, 2011); the nature of the Big Bang (Aretz et al., 2016; Hansson & Redfors, 2006; Prather et al., 2002; Trouille et al., 2013; Wallace, 2011); scales and distances (Coble et al., 2013); and curvature (Coble et al., 2018). These studies have laid the groundwork for this study.

Cosmology and Curriculum

A comparative analysis of 52 curricula across the OECD countries (in addition China and South Africa) has revealed that across primary, middle and high school, there is a certain degree of homogeneity in cosmology topics, with topics in common such as the Big Bang theory; dark matter; dark energy; formation of the elements; the size, scale and age of the Universe (Salimpouret al., 2020a). Although cosmology topics are mainly the province of high school curricula (~ 67%), some concepts such as age of the Universe, scale of Universe and structure of the Universe do appear in upper primary (~ 8%) and lower middle school (~ 25%). Within the Australian National Curriculum, concepts related to cosmology are found in years 9 and 10, while in the Victorian Curriculum, they are present in Year 11 (ibid). Within the Swedish curriculum, topics related to cosmology are found under the overarching subject of Physics in years 10 and 11 (Skolverket, 2020). These curricular patterns justify the surveying of Australian and Swedish years 10, 11 and 12 students’ knowledge of and reasoning about cosmology.

This Study

Since these topics are present in school curricula of many countries around the world, there is a need for better characterisation of the cosmology conceptions held by students and the reasoning that underpins these, that allows us to establish a progression framework to be developed that will usefully inform teaching and learning. This current paper describes the process of how open-ended questions can be unpacked in a way that allows such a progression do be developed, which ultimately informed the development of the Cosmology Concept Inventory (Salimpour et al., 2022). Therefore, this study aims to explore these research questions:

  • (RQ1): What are the conceptions held by high school students with regards to cosmology?

  • (RQ2): What are the forms of reasoning that underpin these conceptions, and how do these relate to the sophistication of conceptions that students hold?

Theoretical Perspectives

A range of theoretical perspectives have been proposed to account for the origin of students’ conceptions and their resistance to change through instruction. Scholars have described them as complex amalgams of intuitive elements (Smith et al., 1994), as complex conceptual systems that organise students’ perceptual experience (Vosniadou, 2002), as ‘mental models’ allowing causal reasoning (Chi, 2008) or as having an internal logic based in everyday reasoning (Andersson, 1986; Watts & Taber, 1996). The theoretical framework for this study draws on the relationship between student conceptions and reasoning (Driver & Easley, 1978). Concepts in cosmology are very often counter-intuitive and removed from everyday experience compared to other astronomical topics like seasons or phases of the Moon, with which at least most students have some degree of familiarity. Driver and Easley (1978), in their work that framed the research tradition around student conceptions, pointed out the power of identifying the reasoning underpinning the vast array of intuitive conceptions that were uncovered in a range of topics. This paper will take this approach in relation to cosmology, in which concepts are very often complex and counterintuitive. Thus, this study aligns with a tradition in conceptions research focused on progression, exploring these conceptions in relation to their variety and the reasoning underpinning these. The aim of this approach is to identify reasoning that can account for progression across multiple dimensions of cosmology where individual studies have explored student conceptions previously, and whether it is possible to use this to develop and validate a cosmology concept inventory. In the field of cosmology, this reasoning can be rich, counter-intuitive and often speculative.

This study uses a combination of nomothetic and ideographic approaches (Driver & Easley, 1978), by making comparisons to the scientific consensus (nomothetic), and also exploring the underlying patterns of reasoning represented by student responses even if they do not conform to the scientific consensus (ideographic). Developing instructional sequences that respond to alternative conceptions requires not only knowing these conceptions but also requires characterisation of the underlying reasoning that produces them.

Methodology

A preliminary earlier analysis of the pilot dataset highlighted the methodological challenges associated with analysing open-ended surveys (Salimpour, et al., 2020b). These challenges included balancing a need to acknowledge the alignment of students’ conceptions with accepted scientific views on the one hand and acknowledging the sophistication of reasoning on the other; dealing with the complexity of language and the imprecision of meaning in unpacking students’ responses, and the associated reductive effect of scoring responses that differed in multiple ways. This study drew heavily on the philosophical underpinnings of qualitative analysis (Creswell, 2011); dealing with responses which were open-ended and richly layered. The approach used thematic analysis to categorise student responses (Xu & Zammit, 2020), with the aim of identifying the underlying reasoning structures associated with different cosmology conceptions. Through this process, the Structure of the Observed Learning Outcome (SOLO) taxonomy (Collis & Biggs, 1979) proved to be a potentially useful framework to make sense of the different levels and types of student responses, particularly utilising the first four levels of pre-structural, uni-structural, multi-structural and relational responses. The current paper takes this analysis further in more closely exploring the variety of students’ conceptions and reasoning using this lens. This also led to the refinement of the survey questions for the second administration, which was implemented a couple of months later but with students from different schools. Refinements included rewording of questions to improve clarity and making the explanation requirement more explicit. In addition, a subsequent analysis using SOLO focusses on the overarching patterns of conceptions at the different SOLO levels within each of the four dimensions of cosmology (size and scale, spacetime location, composition of the universe and evolution of the universe).

Participants

The participants included years 10, 11 and 12 students (ages 16–18 years old) from schools in Australia and Sweden. Random opportunistic sampling was used to recruit schools (Newby, 2014), based on contacts of the research team in the two countries. The final number of students for this study being n (Australia) = 223 and n (Sweden) = 63. Around 80% of students were in grade 10, 15% were in grade 11 and 5% were in grade 12. This is to be expected because in senior years especially in Australia teachers need to cover a lot of content in preparation for exams, making it challenging to get large numbers of students participating in the survey from senior levels. This sampling covered 10 schools, both governmental and non-governmental. The difference in sampling between the two countries is because of differences between school terms that affected when schools were teaching the topics. This study uses Australia and Sweden as instances because both countries have cosmology in the curriculum in grades 10 and 11, some cosmology-related concepts in grade 12, and the authors of the study have experience with both curricula. There was no attempt to compare students from the two countries or compare across year levels, given we have no real basis for expecting differences, and were aiming for theme saturation. Rather, the respondents were treated as a single cohort within which we can expect variety in familiarity with cosmological phenomena, and in reasoning patterns. The survey questions were in English; however, to better articulate themselves, Swedish students could give their responses in Swedish if they wanted. Because these are secondary school students, they have a good grasp of English, therefore, having the questions in English was judged by the Swedish member of the research team to not pose any challenges. The survey had a single question that asked students to rate the amount of Astronomy they had done at school on a Likert-like scale: 0 (Not at all) to 5 (A lot). Overall, most students in each country had some exposure to astronomy previously, up to a ‘moderate amount (3)’ (Fig. 1). The instrument was distributed to the students via an online survey platform, and the data was exported from the survey platform via a comma-separated value (CSV) file.

Fig. 1
figure 1

Visualisation showing student prior astronomy experience based on a 5-point Likert-like scale, with 0 being ‘not at all’ and 5 being ‘a lot’. The y-axis is the percentage of students, and the x-axis is the level of prior astronomy experience

Full size image

The Cosmology Knowledge Survey

The Cosmology Knowledge Survey (CosmoKS) (https://www.untangleduniverse.com/cosmoks/) developed as part of this study is a collection of 28 questions, five of which are multiple-choice, the remaining 23 being open-ended short answer. The multiple-choice questions have a short answer section, where students are invited to explain their reasoning for choosing a particular option. The questions were categorised into four overarching conceptual dimensions (henceforth, dimensions): size and scale, spacetime location, composition of the universe and evolution of the universe (Salimpour et al., 2020b). Through a process of listing the various concepts in cosmology curricula, then through an iterative process of grouping and regrouping, the underlying fundamental characteristics of the concepts were identified as falling into these four dimensions.

The development of the questions took into consideration prior work, combined with curriculum statements for various countries, focussing particularly on Australia and Sweden which were the countries in which the instrument was administered, and the authors’ experience teaching cosmology at both high school and undergraduate levels. The use of open-ended questions allowed for a deeper exploration of students’ reasoning compared to using a set of fixed responses based on consensus disciplinary knowledge. The wider project underpinning the survey design is the development of a concept inventory (Salimpour et al., 2022) designed to capture progression in student conceptions and reasoning, and for this, open ended questions contribute in three ways:

  • Empirically identifying the alternative conceptions students have enables the development of distractors in a concept inventory

  • Student responses which are ‘correct’ can be used as a basis for developing the high-level choices in a concept inventory.

  • Open-ended responses allow for capturing various levels of sophistication and reasoning in student responses. For example, the question: How would you describe our location relative to the Sun? attempts to elicit not simply a position but various levels of sophistication related to how this position can be understood and communicated.

The design of the survey ensured that each dimension had at least one multiple-choice question. Within each dimension there is a subtle substructure in the nature of the questions:

  • Conceptual questions, answers are open-ended, and the concept/knowledge is still debated, for example: Is the Universe infinite? Explain your thinking for your answer

  • Questions requiring reasoning about cosmology research practices regarding concepts/knowledge that is well characterised, for example: How can astronomers determine our location in the Milky Way galaxy? Explain your thinking

  • Declarative knowledge questions with an interpretive addition, for example: What is Dark Energy? How do we know about it?

In the interest of time, and to reduce survey fatigue, each student was randomly allocated eight questions out the 28 initial questions. Each question was on average answered by 30 – 40 students. The randomisation algorithm ensured that the distribution of questions per dimension for each student was equal. A data reduction and visualisation pipeline written in the open-source programming language Python was developed for analysing the data.

Analysis Approach

The multi-stage coding process involved three coders, using an emergent coding approach. The initial coding, carried out by the first author, was based primarily on the type of reasoning within the SOLO taxonomy, with sub-levels reflecting alignment of student answers with canonical ideas, and by the quality of justification. In balancing these different dimensions of response, the first author drew on considerable knowledge of the cosmology field, on knowledge of the conceptions literature, and on knowledge of textbook and public representations of cosmology that can be misleading (Salimpour et al., 2021b). Following this initial coding into levels and sub-levels by the first author, a second member of the team, then went through the first author’s coding and categorisation, and labelled those that they did not agree with. Following this the team members met together and went through the coding, each response was discussed within the team and adjusted as needed to reflect a resolution of any disagreement. The assigning of codes thus corresponded to a theme saturation approach. Interrater reliability measures were not deemed necessary given no statistical comparisons were intended to be performed on these emergent themes.

For example, Question 1: Is the Universe infinite? Explain your thinking for your answer, has a Yes/No/Maybe/Both component (a pseudo-multiple choice) and an explanation component, which was open-ended. Taking both coding slices (components) together, the research team then analysed the responses to determine the lines of reasoning used by the students, i.e. were they using simple declarative knowledge (for example, ‘because of Hubble’s law’), or bringing together multiple lines of reasoning to explain their answer (for example, ‘because Hubble’s law shows that galaxies are moving apart, and the age of universe shows that it is extremely old but it has an age’).

As indicated above, the SOLO taxonomy provided a useful guide to capture a progression in ‘complexity of structure’ (Collis & Biggs, 1979, p. 1) in student responses. The original SOLO taxonomy has five levels, with the highest being Extended Abstract; however, the questions in CosmoKS were not designed to target that level of reasoning. Research has highlighted that the SOLO taxonomy with its five levels does not always capture the intricacy of reasoning perfectly (Watson et al., 1995). The authors of this study recognised this, and so the SOLO taxonomy was used as a framework to guide the construction of a finer-grained set of levels to suit the context of the questions in CosmoKS. In using a combination of nomothetic and ideographic approaches, student responses were analysed both based on their reasoning level, and also alignment to consensus ideas where possible. The reasoning level often but not necessarily broadly aligned with consensus ideas, and where it did not, the lines of reasoning used took precedence in determining the level. The modified rubric consisted of 14 levels and was framed within four of the five SOLO levels (Fig. 2). The subcategories within each SOLO level were based on whether the response was consistent with current scientific thinking, whether it represented an alternative conception, and the quality of elaboration with the response. Responses at the level 12, for instance, would link at least two ideas in a valid way, would be scientifically defensible and justified, but not refer to a general principle. It should be noted that the difference in responses at level 13 and level 14 is based on the completeness/correctness of responses, those at level 13 are not necessarily always fully ‘correct’ (Fig. 3). The construction of these fine-grained distinctions within the broader SOLO levels provided insight into the richness and complexity of students’ reasoning and conceptions. At the uni-structural level, for instance, there were many student conceptions that would have been collapsed within a single code, that we were able to capture. The fine-grained divisions within uni-structural represented the combination of different reasoning and conceptions including their ‘correctness’, how declarative knowledge was used, the interaction between declarative knowledge and pre-empirical explanations, such that we were capturing real differences even though these were very context dependent. It should be noted that contextual variations in the types of questions and the knowledges they represented meant that the finer-grained levels did not necessarily manifest for all questions and were only applicable to certain ones.

Fig. 2
figure 2

The pattern of levels across the broad SOLO Taxonomy categories. The evidence for demarcating between the levels within each SOLO category was based on a combination of the validity of the answer and/or the supporting explanation

Full size image
Fig. 3
figure 3

Example of student response for some of the levels described in Fig. 2

Full size image

In order to illustrate how judgements were made with regards to categorising student responses and coding them according to the fine-grained 14 levels, three questions were selected, each from a different dimension to show how the relationship between reasoning and conceptions was unpacked, and illustrate how this balance was achieved in the coding (Fig. 3). The examples also highlight the complexity and richness in the reasoning that is captured within each of the SOLO levels.

The sub-levels within each of the SOLO levels took into consideration the ‘correctness’ of the answer, in essence to what degree the answer aligned with consensus views, and also the level of reasoning. The development of these sub-levels was done through iterative coding where the responses were first sorted into correct/incorrect, these were then analysed to unpack the reasoning, which was iteratively coded until overall emergent codes were identified.

Results

The questions were sorted according to total score based on the numerical value for each of the 14 levels. This allowed each question to be explored individually, as well as comparing the questions across the various dimensions. However, the complexity of responses made it difficult to see patterns across the data, so for this purpose the data was collapsed back into the four SOLO levels. This allowed a better perspective on the patterns of reasoning across the individual questions and dimensions. Figure 4 provides an overview of the number of responses at each of the SOLO levels.

Fig. 4
figure 4

Visualisation showing the total number of student responses at each of the four SOLO levels, across the 28 questions

Full size image

The collapsed levels reveal that most of the student reasoning is within the pre-uni-multi-structural rather than relational region. Looking at the overall visualisation in Fig. 4, a left-skewed Gaussian distribution is evident, showing a strong pre-structural presence. This is perhaps not surprising given that for some questions students may not have been exposed to the content. To understand this distribution in terms of the nature of student reasoning about phenomena that often complex and counterintuitive requires probing deeper into particular questions. Figure 5 shows the distribution of reasoning levels for four questions selected to represent variation in the structure and type of reasoning they trigger. Three were high scoring questions with many students reasoning at multi-structural or relational level, which meant students had a basis on which to reason, and one was lower scoring (question 18) but selected to provide variety in question type and dimension. It can be seen that each of these has a definite but different pattern of response.

Fig. 5
figure 5

Visualisation showing the total number of responses in the four SOLO Levels for four questions different types of questions, three of which were high scoring: Question 20: How has the temperature of the Universe changed over its lifetime? Explain your reasoning why you think this is so—(a) It has increased/become warmer; (b) It has decreased/become colder; (c) It has not changed/remained the same; (d) I do not know. Question 11: How would you describe our location relative to the Sun? Explain your thinking. Question 15: What was created in the early stages of the Universe? Explain why you think so. Question 18: What is Dark Matter? How do scientists know about it?

Full size image

The different question structures and dimensions uncover the range of reasoning around conceptions evident in the responses. These differences in response patterns for these questions will be evident from the analysis. To better understand the clustering for each of the questions required not only identifying student responses within each of the SOLO levels but also exploring the underlying conceptual and reasoning commonalities in student responses. These were identified by iteratively coding student responses for each question to identify over-arching themes. The construction of these themes is also evident in the analyses below.

Reasoning Patterns of Students

Question 11 (located in the dimension spacetime location) asked: How would you describe our location relative to the Sun? Explain your thinking. The noticeable peak at uni-structural level (Fig. 5) reflects a majority of responses that were presentations of straight facts, for example, uni-structural responses mentioned ‘third planet from the Sun’, or ‘1 AU’.

Question 15 (located in the dimension composition of the universe) asks: What was created in the early stages of the Universe? Explain why you think so. It can be seen, in Fig. 5, that the responses are evenly spread across three of the four levels. Student responses can be broadly categorised into those that consider:

  • Astronomical objects like stars, planets, because these are associated with or necessary for life without attending to the evolutionary process, located at the pre-structural level

  • Hydrogen and helium, because they are the most abundant elements, and it is what stars are mostly made up of, located at the uni-structural level

  • Particles, for example, protons and neutrons because they make up the elements, and elements are the building blocks of everything, representing two-step reasoning, therefore, it is located at the multi-structural level.

Question 20, the highest scoring question, asking how the temperature of the Universe has changed over its lifetime, illustrated that similar reasoning could result in opposing answers. The majority of students were distributed equally between two options: thinking the temperature has (i) increased and (ii) decreased. The reasons students provided, despite some being alternative conceptions, drew on and connected several ideas, and are therefore clustered in the multi-structural level. For example, those who reasoned the temperature of the Universe has increased attributed this to climate change, perhaps reflecting media headlines connecting temperature increase with climate change. It also relates to a more fundamental challenge where students perhaps consider the Universe from a geocentric perspective, linking ‘global’ with ‘universal’. In the case of students who selected that the temperature of the Universe has decreased (thermodynamic intuition), there were alternative conceptions about the cause of this decrease. For example, ‘as the space between the stars was increasing therefore the heat is not sustained’. In this case, it is evidence once again of the difficulty in appreciating the size and scale relations of objects within the Universe.

The open-ended questions which were purely explanatory required extracting themes from the answers, often reflecting multiple alternative conceptions.

Question 18: What is Dark Matter? How do scientists know about it? is an example of a question dealing with both declarative and epistemic knowledge, testing students’ definitional knowledge, and of how knowledge is created in science. The coding process extracted 15 themes across the four SOLO levels (shown alphabetically in Fig. 6). There are some noticeable peaks in the distribution representing common ways of reasoning.

Fig. 6
figure 6

Visualisation showing the themes extract from student responses to question 18: What is Dark Matter? How do we know about it? This question is situated in the composition of the universe dimension. The themes are grouped based on the SOLO levels

Full size image

These themes highlight a range of alternative conceptions, for example, associating dark matter (DM) with blackholes, empty space, antimatter and also more importantly the confusion between dark matter and dark energy (DE). These single lines of reasoning responses are grouped in the uni-structural level. The confusion between dark matter and dark energy is also prevalent in question 17, which asks: What is Dark Energy? How do scientists know about it? A large percentage of students were not able to answer the second part of the question or used answers which are essentially at a pre-/uni-structural level such as ‘I saw it on the internet’, ‘I read about it on the internet’, ‘My teacher told us’, ‘through research’, ‘using advanced telescopes’. Some students were able to point out detection via gravitational influence; however, they were stated as simple facts.

At the multi-structural level, students are highlighting key characteristics about DM being a form of matter, not emitting light and being detectable by gravitational influence. At a multi-structural level, students have varied conceptions that are based on connecting intuitive experiences and ‘exciting facts’.

Many responses to the dark energy question linked energy with force, perhaps having roots in everyday language and experience (Megalakaki & Thibaut, 2016): ‘It is a mysterious force; DE is a negative form of energy that repels, rather than attracts’.

Overview of the Conceptions and Reasoning Across the Dimensions

Synthesising the coded themes into overarching conceptions allows us to look at the location of common alternative conceptions in the four dimensions and to gain insights into what type of reasoning underpins these alternative conceptions. Figure 7 shows the distribution of alternative conceptions across the four dimensions. Some conceptions overlap between dimensions, demonstrating how understandings in cosmology can require creating conceptual connections between these dimensions. Situating the conceptions within the dimensions provides insights into deeper and specific patterns of reasoning that may otherwise not be apparent in a simple list. These four dimensions are also fundamental in that they can be applied to different domains in Astronomy and are not limited to Cosmology.

Fig. 7
figure 7

Distribution of alternative conceptions within the four dimensions

Full size image

One aspect that this study aimed to explore was how the levels of reasoning represented by the SOLO levels would be situated within each of the four conceptual dimensions. We undertook an exploratory analysis by taking a slice through the data looking at the distribution of student responses at each of the SOLO levels across the four dimensions (Fig. 8). Within the dimension spacetime location, there is a strong clustering at the uni-structural level. This dimension tends to be about spatial relations, which students may know but lack a deeper insight into, for example, Earth being the third planet. In size and scale, questions tended to explore limits of current cosmology knowledge based on observational evidence, thus prompting speculative reasoning, there is a clustering within the uni-structural/multi-structural region. In composition of the Universe, there is a dominant distribution of responses within the pre-structural region, and a relatively equal clustering in the uni-structural/multi-structural region, albeit much lower than pre-structural. This strong clustering in the pre-structural relates to students having alternative conceptions of DE and DM, and how knowledge about these properties of the Universe is obtained. In evolution of the universe, the distribution is more dominant at pre-structural/uni-structural region, because students often ‘know’ things but are not really thinking beyond intuitive interpretations of the Big Bang, expansion, etc. or making connections with regards to interrelated concepts across the four conceptual dimensions.

Fig. 8
figure 8

Visualisation showing the distribution of the percentage of student responses for each of the four SOLO levels versus the four dimensions

Full size image

Nature of Conceptions and Their Relation to the Four Dimensions

The above analysis focussed on the sophistication of reasoning in the responses, drawing on insights from the extensive body of work on student conceptions. Reflecting Driver and Easley’s (1978) seminal finding that student conceptions reflect more than merely a progression through Piagetian stages, we expect patterns in conceptions to reflect context-specific levels. To explore these patterns, the next section extracts key themes in students’ conceptions within each of the four dimensions. For this analysis, all responses at each of the SOLO levels within each dimension were gathered to thematically extract the conceptions associated with these. The analysis followed the same approach as used for individual questions; an initial coding was conducted by the first author, followed by iterative cycles of refinement with the research team ensuring inter-coder reliability.

Size and Scale

In the dimension of size and scale, student reasoning varies in how they navigate the enormous size and scales involved. There are students who reason that because the Universe is huge, it must be infinite and so this creates a conception of never-ending edgelessness, for example, ‘I think the universe could be infinite. As space goes forever I feel this is true’. However, students also use the fact that the Universe is expanding to place limits on the size and scale of the Universe—reasoning that expanding objects are limited, for example ‘No, the universe is not infinite. Although it is continuously expanding, it cannot expand forever, as eventually matter will run out’. These responses use multiple lines of reasoning, albeit the deductions are different, and are situated at a multi-structural level. There is also evidence in student responses of confusion between the Galaxy and the Universe, for example ‘I don’t think the universe doesn’t have an edge, the Universe is so large that it is hard to find the exact location of the ‘last’ star in the universe. I think that the universe gets to a point where the stars are so spread out that the stars are light years away from each other’. This is made evident when students believe that expansion is happening on the scale of galaxies, for example, ‘It is constantly expanding—an edge reveals flat side whereas the galaxy is circular…’.

In terms of scales, students reasoned that because an object is large everything about it must be large, for example, ratio of distances to sizes is large for the biggest objects in the Universe ‘Because Galaxy cluster is many Galaxies and therefore is bigger than a galaxy’. Students are challenged to appreciate the scales, but also to move between scales. Previous studies have identified this as one of the challenges in astronomy education (Coble et al., 2013; Lelliott, 2010; Miller & Brewer, 2010; Rajpaul et al., 2018).

Spacetime Location

In the dimension of spacetime location, there are related challenges for students reasoning in non-tangible 3D space, identification of which can inform the pedagogy that may be required to help them surmount these. There is evidence of a confusion between galaxy, Universe and the solar system, with some students using the terms interchangeably, for example ‘Our universe is placed in the Milky Way, third planet from the sun.’ or ‘On the edge of out universe. Not the very edge but closer to the edge than the black hole in the centre of our universe’. Overall, students have difficulty moving between reference frames and the difficulty of extrapolating to larger scales further compounds the issue.

In addition, although students may know disciplinary terms, they are not able to effectively reason using the underlying concepts and the evidence base for these. For example, How can scientists determine the approximate age of the Universe based on observations of the speed of many galaxies? Explain your thinking; elicited the response ‘Hubble’s constant’. Responses of this nature are uni-structural in that they are mere associative snippets, without depth in explanation.

Composition of the Universe

In the dimension composition of the Universe, students reason that most of the Universe is empty because of the emptiness of space, for example ‘Empty space: Space was named space because it mostly consisted of nothing’. Responses of this nature are situated in the pre-structural level, since students are drawing on intuition.

There is evidence of a confusion between notions of DM, DE and the intuition regarding ‘dark’ being something that is unknown. For example, ‘Dark energy is the matter that is beyond the universe which the universe is expanding into. So far, dark matter has unknown properties…’. Students also relate the notion of ‘dark’ to black holes, for example, ‘I would imply that Dark Energy is the strong force associated with black holes’ or ‘Dark matter is the unknown factor of a black hole’. Although students may know key terms, they have difficulty in connecting this disciplinary terminology to concepts, for example, ‘I have heard this term before in an Astrology message I was a part of but I do not remember what it is exactly’.

Evolution of the Universe

In the dimension of evolution of the Universe, there were questions based on factual knowledge; however, they required students to explain their reasoning. Students are sometimes familiar with the key terms and facts; however, they do not reason using those terms. There is a disconnect between facts and concepts, for example, How has the temperature of the Universe changed over its lifetime; student response: ‘It has increased/become warmer: (second?) law of thermodynamics, entropy always increases, leading towards the heat death of the universe’ or the two-part question: What is the Cosmic Microwave Background Radiation? What is its significance?; student response: ‘The big bang theory’ to both parts. These types of responses tend to be situated at the uni-structural level, or at times the pre-structural level. The association of ‘heat death’ with warming is another example of intuitive associative reasoning.

Students also do not show an appreciation of the scale of expansion, consistent with similar reasoning for the dimension size and scale. Students have the conception that the Big Bang is the origin of everything, being a physical explosion in space, for example, ‘The big bang is a big explosion that created stars and planets and has created life on earth and maybe on other planets’; ‘the big bang theory is the beginning of the universe’; ‘The Big Bang Theory is the leading explanation about how the universe began. At its simplest, it says the universe as we know it started with a small singularity, then inflated over the next 13.8 billion years to the cosmos that we know today’. These responses are at the uni-structural level, as they involve simple statements of fact, often representing alternative conceptions, rather than involving any chain of reasoning. This conception is consistent with inaccuracy in representations of the Big Bang in the media and textbooks, which students are perhaps drawing on to reason.

Discussion and Implications

Students in both Australia and Sweden gauged themselves as having a similar, ‘moderate’ exposure to astronomy. However, the survey responses and analysis of reasoning suggest that having prior exposure to astronomy and knowing about the Big Bang does not guarantee that students will have the conceptual understanding required to navigate the complex ideas encompassed within topics of cosmology. Figure 8 shows the distribution at the relational level is quite low and strong clustering at the uni-structural level.

Identifying the Reasoning Challenges Underlying Student Responses

The various alternative conceptions highlighted in this study align to findings in previous work (Aretz et al., 2016; Wallace, 2011), for example the notion of the Big Bang being an explosion or a point in space. In this study, however, we have attempted to identify patterns of reasoning that provide deeper insights into the source of these conceptions, across the four dimensions of cosmology. Characterising how these patterns develop has implications for the teaching and learning of cosmology, and science education in general.

This study identified some fundamental patterns in student reasoning about cosmology, both across and within the SOLO levels. Students may not have a mental map of the relationship between entities, exacerbated by the difficulty in comprehending the scales. For example, confusing galaxy, Universe and solar system stems from a confusion about entities within large scales and relationship between them (Rajpaul et al., 2018). Students may know that the galaxy is 100,000–120,000 light years in diameter; however, what does that mean in relation to everything else in the Universe? Thinking that expansion is happening on the scales of stars and individual galaxies also reflects difficulties in navigating the spatial relations between systems. Students describing the Big Bang as an explosion, or a physical spatial point are leveraging their language and experience with the concept of ‘Bang’. Thinking that the Big Bang is about creation and origin is also related to language, experience and how the Big Bang is portrayed in visualisations (Salimpouret al., 2021b). Furthermore, it also relates to how the human mind thinks about evolution through a linear temporal lens; there needs to be a beginning. Looking at disciplinary definitions, students know about Hubble’s law (now called Hubble-Lemaître law), or that distant galaxies are moving faster, but they do not appreciate the import of these concepts/phenomena. The same is evidenced in student responses related to dark matter, dark energy, inflation and related disciplinary terminology. Students know something of the terminology and principles, but their explanatory language indicates a misunderstanding of logical connections between theories and evidence. A grasp of cosmology even at a descriptive non-mathematical level requires the coordination of a range of concepts and evidential trails.

In essence, we can summarise the fundamental reasoning challenges as:

  • Navigating spatial and temporal relations which represent huge and hard-to-grasp numbers, and difficult space-time relations that offend everyday logics. Students have difficulties not only with the huge scales of space and time, but this is also coupled with confusion about how entities relate to each other—the universal vs local and what is part of what.

  • Counterintuitive concepts. Quite a few difficulties relate to counterintuitive concepts such as curvature, expansion, the edge of the Universe, where using normal reasoning does not work. These difficulties are compounded by the challenge of representing such complex spatial and temporal happenings in public and textbook representations.

  • Language and everyday experience, especially intuition. This is related to the problems of intuitive and associative thinking (warming = global = Universe, dark = invisible, etc.) More importantly, the use of intuition and associative thinking which can lead to student alternative conceptions can be very resistant to change (diSessa, 2004; White & Gunstone, 1989), particularly when scientific concepts are contrary to everyday experience, as is the case with cosmology.

Students seem to reason that a well-established theory is in and of itself evidence. For example, when asked: What evidence is used in support of the Big Bang Theory? Students tended to describe the Big Bang theory, rather than the evidence. On a related reasoning pattern, students were able to state facts or the scientific terms; however, they had difficulty connecting the terms to an evidential or conceptual framework. For example: Explain in your own words what is meant by the statement: The Universe is expanding on a large scale; student response: ‘Due to the Hubble’s law and the cosmic microwave background both proves that universe is expanding on a large scale’. Another example, Scientists say that the early universe went through a period of inflation. To what are they referring?; student response included ‘In principle, the expansion of the universe could be measured by taking a standard ruler and measuring the distance between two cosmologically distant points, waiting a certain time, and then measuring the distance’. These instances would be situated in the uni-structural level of the SOLO taxonomy.

Students when presented with scenarios that invites reasoning would sometimes state facts or ideas that they find fascinating, albeit being partially understood or perhaps not relevant to the question. For example, when asked, It is predicted that the Andromeda Galaxy and our own Milky Way Galaxy are on a collision course. How can this be the case if the Universe is said to be expanding?; student responses included, ‘I know that the further away something is from us, the faster it is moving away from us.’; ‘I think the universe will have a Big explosion’. Or for the question, What was created in the early/initial stages of the Universe’s evolution?; student responded with ‘I believe in evolution because there is proof in whales and other animals’. These examples indicate a lack of a structure, or ‘fracturing’ in students’ knowledge.

This is perhaps in part due to the popular representations of exciting facts in cosmology, which despite being engaging, amazing and intriguing, owing to their counter-intuitive nature can be misinterpreted and misrepresented in the media. The fascination that Astronomy holds for people, associated with the many counterintuitive or strange phenomena that are reported in textbooks or other media, can arguably promote the acceptance of such phenomena without them being embedded in coherent frameworks of understanding. This meant that many responses to questions did not address the logic of conceptual connections or connecting of ideas to evidence. In essence, the ‘wow’ effect can detract from depth of understanding. For example, Is the Universe infinite? Explain your reasoning; ‘Yes, because there is more than one universe’. Clearly the student is familiar with the concept of multiverses, but this leads to confusion.

Research has highlighted that cosmology representations in textbooks can be misinterpreted (Ojala, 1992; Salimpour, et al., 2021b; Testa et al., 2014), that concepts in cosmology can be inaccurately portrayed in the media (Davis & Lineweaver, 2004), and that the manner which these concepts are framed in curriculum can be misleading (Salimpour, et al., 2020a). For example, representations of the evolution of the universe that show the Big Bang as a point in space (Salimpour et al., 2021b). Student responses from this survey provide some indication of this, for example, ‘I read it in a book’ or ‘I watched in on the internet’. This creates a problem especially for teachers who have no or limited background in the topics and without support are liable to unintentionally teach those alternative conceptions. When it comes to topics such as cosmology merely including statements in the curriculum as a form of ‘engagement’ does not make use of the potential of topics. We recommend that curriculum developers need to situate these topics in a progression of concepts, rather than isolated instances, this has been discussed in recent work (Salimpour et al., in prep).

Characterising Student Reasoning

Bringing together the findings from the analysis in this study, it is possible to show the characteristics of reasoning across the four dimensions using the SOLO levels. This was made possible through the slicing and coding of the student responses across dimensions. In this section, we discuss the insights into the various conceptions and their bases, and the hierarchy of sophistication in reasoning that relates to these conceptions. While the format of the survey may have inhibited students from adequately representing their reasoning such that we cannot claim to have captured their thinking fully, we argue that the identification of patterns across 200 responses provides some confidence in the analysis. The SOLO taxonomy provides a framework for characterising this hierarchy, and particularly what characterises students’ high level, relational reasoning. At the relational level, student responses demonstrate not only a sense of understanding of the concepts being probed, but also the import or wider significance of those concepts. For example, for the question: How would you describe your location in the Solar System?; knowing that the Earth is the third planet from the Sun is a minimal response. Students reasoning at a relational level will appreciate the wider significance of this on habitability. Similarly, knowing that the cosmic microwave background is related to the Big Bang or a glow from the early Universe is fine; relational reasoning would recognise the significance of the cosmic microwave background. It should also be highlighted that to really understand and appreciate the concepts in cosmology at a relational level includes appreciating the evidential setting of the concept and the historical significance in the field. It is important from a disciplinary perspective, especially perhaps for the field of cosmology to understand how knowledge is built over time.

Students at the multi-structural level identify some aspects of the concepts but do not bring them together into a coherent line of reasoning. Tytler (1998a) found that students often apply their conceptions inconsistently. The uni-structural level is characterised by the use of facts as isolated knowledge snippets without an appreciation for how they fit into the field of cosmology—indicative of what we term ‘fractured knowledge’. At the pre-structural level, there is a lack of appreciation and understanding of the various concepts, for example the size and scale relations. From these analyses based on the SOLO levels, particularly identifying the types of student conceptions and their reasoning, we intend to build a refined instrument to construct a conceptual progression for each of the four dimensions. This will be explored in a follow-up study.

It is important to highlight that some of the reasoning leading to alternative conceptions can be quite sophisticated and can be harnessed to support conceptual growth through guidance towards consensus ideas (Tytler, 1998b). Pedagogical resources are needed that can be used to scaffold students in following lines of reasoning rather than grasp isolated facts about cosmology. This study has attempted to understand patterns of conceptions in terms of student reasoning.

Conclusions

The key aim of this study was to provide insights into how students reason in cosmology topics. In answering RQ1, What are the conceptions held by high school students with regards to cosmology? Based on thematic coding of responses to an open-ended survey this study has identified a range of conceptions held by students that can be related to specific reasoning challenges, across the four conceptual dimensions of cosmology (size and scale, spacetime location, composition and evolution of the universe). In answering RQ2, What are the forms of reasoning that underpin these conceptions, and how do these relate to the sophistication of conceptions that students hold? This study has synthesised the underlying reasoning challenges for the various alternative conceptions into three overarching patterns: navigating spatial and temporal relations, counterintuitive concepts and language and everyday experience. In doing so, this study has demonstrated relations between reasoning levels and conceptual sophistication. The results show that even high-level responses are not necessarily correct and complete. This research has identified the SOLO taxonomy as providing a reasoning framework that can account for student conception progression across multiple phenomena that have previously been studied separately. The findings were used firstly to develop a more targeted concept inventory (Salimpour et al., 2022) that will translate these reasoning levels into a progression map of cosmological concepts. Secondly, we intend to develop teaching and learning resources that can support students in reasoning about key cosmology ideas.