Abstract
This multi-phase study designed a program evaluation rubric which used a qualitative rating system to help teachers identify key learning goals, dimensions, and principles related to the socio-scientific issues (SSI)-based science, technology, engineering, arts and mathematics (STEAM) approach for science education. This rubric contains 37 criteria that address one or more key principles of the SSI-based STEAM approach embedded in a 6E inquiry-based instructional model. It is organised into seven evaluation categories: (1) learning outcomes, (2) engage, (3) explain, (4) explore, (5) elaborate, (6) evaluate, and (7) enact. The newly developed rubric was used to evaluate six different climate change programs to determine whether it could measure the intended dimensions and principles and to assess its applicability across varying contexts. This research has implications for providing a tool for teachers to plan and design SSI-based STEAM programs.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
Introduction
Many science curricula emphasise the need to contextualise science in real-world problems to help make science learning more relevant and meaningful for students (Goodrum & Rennie, 2007). The ability to critically engage with and address these real-world problems has been increasingly important as students face new and existing socio-scientific issues (SSIs) such as climate change, air pollution, sustainability, and genetic engineering. This emphasis has shifted into the forefront as science education places greater priority on promoting citizenship as an educational goal, rather than just focusing on student engagement, science knowledge, and skills development (Sadler, 2011). To address this need, researchers and educators are turning towards implementing alternative educational approaches like science, technology, engineering, arts and mathematics (STEAM). STEAM education integrates arts-related practices and perspectives (e.g., humanities and intercultural learning) to provide a different lens for students to interpret and understand science and comprehend how things in the real world relate to each other (Herro et al., 2017; Yakman, 2008). The inclusion of the arts brings in creative modes of learning and reframes the learning through the social and humanistic dimensions inherent in science (French, 2020; Abed, 2016). However, as STEAM-based education has continued to evolve to meet the demand that students need to understand systems and connections more deeply in relation to real-world issues, a new conceptualisation of the STEAM approach has emerged (Mang et al., 2021; Yakman & Lee, 2012).
SSI-based STEAM is a relatively recent interdisciplinary approach which integrates SSI into STEAM practices (Mang et al., 2021). By merging SSI perspectives with STEAM, teachers not only create exciting and creative learning opportunities, but also connect science learning more authentically and meaningfully to students’ everyday lives (Baek et al., 2022). This approach is more likely to allow students to see the purpose of science in their lives and thus encourage them to develop a stronger sense of responsibility and willingness to enact changes and new behaviours to address real-world issues (Alcaraz-Dominguez & Barajas, 2021; Choi et al., 2021). Studies have shown that SSI-based STEAM fosters the development of students’ conceptual knowledge, skills, and practices, as well as positive values and attitudes (Baek et al., 2022; Chu et al., 2019). By focusing equally on the development of all these components, teachers can help to enhance students’ scientific literacy (Mang et al., 2021). According to Lin et al. (2021), the use of STEAM could also reduce the gender-based interest and participation gap and differential participation in industrial development seen between girls and boys in the sciences.
SSI-based STEAM evolved from the awareness that STEAM as a standalone approach had shortcomings that impacted on its capacity to be implemented deeply and effectively in the classroom (Mang et al., 2021). For example, interdisciplinary connections are often superficially addressed in classroom activities, which makes it difficult for students to authentically engage with real-world problems (Herro et al., 2019). The lack of engagement with real-world problems becomes more apparent when the issues selected by teachers tend to lack complexity and social, cultural, personal, and historical relevance (Zeidler, 2016). These issues can be addressed by integrating SSI into STEAM practices and content. This is made possible because both approaches have overlapping elements such as the emphasis on inquiry-based learning, affective learning, and interdisciplinary learning (Fig. 1; Mang et al., 2021; Klosterman & Sadler, 2010). These intersecting elements found in SSI and STEAM could complement and help strengthen learners’ experiences with more connected science learning.
Although there are overlapping elements, it is important to note that the pedagogical application of SSI-based STEAM is still distinct from SSI and STEAM. Table 1 provides a comparison of the key foundational characteristics of all three approaches and how they are adopted within the classroom. Generally, the SSI approach focuses on incorporating the intersection of science and society so that students can consider socio-political, cultural, moral, and ethical views that may enhance the real-world application of their scientific knowledge (Ekborg et al., 2009). In doing so, SSI teaching often adopts open-inquiry strategies to enable students to deal with science content framed within a problem/case that is both controversial and open-ended in nature (Klosterman & Sadler, 2010). The aim is to allow students to develop their own position on the SSI topic based on their investigations, and thus empower them to become activists on social and environmental issues (Klosterman & Sadler, 2010; Reis, 2014). The STEAM approach focuses on using the inherent connections between science and other disciplines as access points for eliciting students’ inquiry, interest, and curiosity to solve real-world problems (Hong, 2021). Therefore, inquiry-based learning strategies typically focus on positioning students in inquiry processes that provoke creativity, encourage exploration, and emphasise affective learning (Park & Park, 2020).
Some common features such as contextualising science concepts in real-world problems can be seen at varying degrees across SSI, STEAM, and SSI-based STEAM. However, the way the common features are executed in the instructional process in each approach is different. SSI-based STEAM distinguishes itself from the other approaches by utilising a structured inquiry-based learning approach to guide SSI-mediated STEAM lessons. Through this instructional approach in SSI-based STEAM, students apply science concepts to study real-life social scientific issues and engage in action to address the issue. This is different from STEAM, where although science learning is situated in real-world problems, the problems need not be social scientific issues. SSI-based STEAM is distinct from SSI in that SSI focuses on social scientific issues, and while students do learn science in the process, the primary aim is to raise awareness of the issue and help students develop a position regarding it. Furthermore, the emphasis in this approach is on providing concrete, practical opportunities that teachers can plan within the classroom context to allow students to enact the knowledge, values, and practices they develop (Mang et al., 2021), enabling students to continually practice using and understanding the interconnected roles of knowledge, values, and practices when decision making and problem solving, which will better support their capacity to become global citizens who can address science-related social problems (Clément, 2012; Lee et al., 2013).
Many teachers support and value the importance of SSI-based STEAM; however, they often struggle to conceptualise how to interpret and translate STEAM practice into their classroom (Quigley & Herro, 2016; Won et al., 2021). This tends to lead to teachers designing and implementing activities that address content superficially and in a disjointed manner (Quigley & Herro, 2016; Rodrigues-Silva & Alsina, 2022). Furthermore, teachers struggle to engage in assessment strategies that evaluate the overall learning process rather than just the outcome or product (Cook et al., 2020). The challenge arises because teachers typically lack knowledge, experience, and understanding of SSI-based STEAM and therefore often are reluctant to implement SSI-STEAM practices (Won et al., 2021). This mismatch between teachers’ perceptions and their STEAM practices is often rooted in teachers’ misconceptions and biases about the approach and the science topics that may be targeted in an SSI and STEAM-based curriculum, as well as the perceived lack of professional development (PD) opportunities for teachers applying SSI-STEAM (Boice et al., 2021; Kind, 2019). Currently, there are some theoretical models such as Baek et al.’s (2022) SSIBL-STEAM and SSI-based STEAM (Mang et al., 2021) that are available to help teachers understand the theory behind the proposal to integrate SSI perspectives into STEAM. With the addition of PD opportunities, teachers could develop a stronger understanding of the benefits of adopting STEAM as a teaching approach (Herro & Quigley, 2016). However, studies such as Boice et al. (2021) and Cook et al., (2020) found that while STEAM PD improved teachers’ knowledge and practices, they still required ongoing support to identify and plan STEAM lessons and encourage the actual practices. For example, Herro et al., (2017) developed a rubric called ‘co-measure’ to help teachers and educators assess student collaboration in STEAM activities. The development of similar programs or assessment tools in SSI-based STEAM would provide stronger support and incentivise teachers to implement the approach. At present, it appears that there are no quantifiable standards available to help teachers identify the overall components of SSI-based STEAM such as task-relevant assessment strategies, selecting meaningful SSI contexts, or ways to meaningfully include interdisciplinary and integrated practices (Cook et al., 2020; Won et al., 2021). Furthermore, there are only limited examples of such SSI-based STEAM programs available; as a result, teachers would need to rely on their own intuition to determine whether the programs already adopted are effective, and how they could be revised for future use. The study presented here is part of a joint project between researchers in Australia and Korea, who have collaboratively developed a program evaluation rubric to help teachers plan and assess SSI-based STEAM programs used for science learning.
The SSI-Based STEAM Approach Model and the 6E Inquiry Model
In the context of this study, the authors’ SSI-based STEAM approach will be adopted. According to Mang et al., (2021), teachers implementing the approach need to understand three dimensions (Fig. 1): (1) learners as active participants, (2) the four key principles (enacted values and practices, affective learning, authentic context and activities, and interdisciplinary thinking and integrated practices), and (3) three educational pedagogical approaches (social constructivist-, situated-, and inquiry-based learning). To translate this theory into practice and identify both the STEAM and SSI components, teachers need to adopt the 6E inquiry instructional model and engage students with six stages of inquiry: engage, explore, explain, elaborate, evaluate, and enact (Appendix, Table 5). The 6E inquiry model adapts Bybee et al.’s (2006) 5E inquiry model by including an additional inquiry stage known as ‘enact’. In the ‘enact’ stage, there is a focus on giving students the space to have meaningful discourse and dialogue about different socio-cultural and political values and views, and how they affect social practices. Students are then given opportunities to put into action the values and practices they have learned. This inquiry model helps guide teachers in relinking the science concepts and activities back to the SSI topic, emphasises the interdisciplinary dimensions, and allows students to use their own thoughts, values, and emotional experiences to mediate their learning and their exploration and negotiation of different identities that they may adopt in their lives (Gao et al., 2019; Mang et al., 2021). This fosters stronger contexts for students to understand how values, identities, and emotions affect the way knowledge is used in practice and decision-making (Lee & Brown, 2018; Sjöström et al., 2017).
With the lack of available content and pedagogical knowledge or experience of SSI-based STEAM teaching, it is crucial that instructional support such as program evaluation tools be accessible (Won et al., 2021). Such evaluation tools can help teachers to gauge the extent to which a program reflects the approach and targets related skills and content. This in turn ensures teachers are designing appropriate learning opportunities that explicitly reflect the core dimensions and principles of SSI-based STEAM and can make adjustment to the programs, materials, and instructions (Herro et al., 2017). Not only can this enhance students’ learning outcomes, but it can also support teachers’ successful implementation of SSI-based STEAM practices. Currently, there are no program evaluation tools available to assess the quality and scope of SSI-based STEAM programs.
The Need for a Program Evaluation Rubric
When developing and implementing programs in the science classroom, teachers need to carefully consider whether their program constructively aligns with relevant outcomes, goals, curricula, educational theories, and instructional strategies (Centre for Education Statistics and Evaluation, 2015). To do this, teachers need to have adequate knowledge about the program and curricular and pedagogical practices (Kind, 2019). SSI-based STEAM PD or teacher training is one way to help teachers improve in areas such as using inquiry-based instruction and planning appropriate lessons (Rodrigues-Silva & Alsina, 2022). As teachers continue to develop their capacity to plan lessons using unfamiliar approaches like SSI-based STEAM, it is vital that ongoing support is provided after the training is finished, to facilitate their further learning and practice. In Boice et al.’s study (2021), teachers were provided additional support after their STEAM PD, such as access to materials, lesson modelling, and weekly newsletters with resources, and meetings. However, to continue to support teachers to determine and evaluate the quality of their SSI-based STEAM programs or materials, program evaluation tools like rubrics are vital.
A rubric is an evaluation tool that sets out the criteria that reflect dimensions of interest and are used to make a judgement about the performance or quality of work done (Martens, 2018a, b). The use of rubrics will ensure greater transparency during the evaluation process and allow teachers or researchers to determine whether the program meets targeted outcomes, and thus clarify what SSI-based STEAM program quality and success look like (Dickinson & Adams, 2017; Wilkerson & Haden, 2014). The inclusion of a program evaluation rubric during program planning provides opportunities to revise throughout the process and helps teachers to justify how well their own developing or existing programs reflect the approach (Martens, 2018a, b; Wilkerson & Haden, 2014). Furthermore, the evaluation rubric could be used to improve teachers’ teaching and learning practices by helping them to document program and activity development during program planning and to ensure that they consistently apply the approach for student learning (Kind, 2019).
Climate Change as an SSI Topic
In this study, climate change was selected as the SSI topic of interest for program evaluation. It is a current global issue, with catastrophic impacts being observed on aspects such as food security, biodiversity, wellbeing, weather patterns, and sea levels (Intergovernmental Panel on Climate Change [IPCC], 2018). The work presented in this paper was adapted from a Master’s thesis study conducted in 2019–2020. At that time, Australian students experienced extreme environmental events including droughts, water restrictions, bushfires, intense hailstorms, severe thunderstorms, and cyclones (Australian Bureau of Statistics [ABS], 2020). In the state of New South Wales, students experienced the ‘Black Summer' fires, the worst bushfire season in state history, with more than 19 million hectares of land burnt, over 3000 houses destroyed, and 3 billion animals killed or displaced (Filkov et al., 2020; NSW State of Environment, 2021). Therefore, climate change was an appropriate topic to select for the purposes of this study due to it being a compelling and emotional event that students were experiencing and have since continued to experience.
Although students tend to be aware of climate change, research has indicated that many students have poor knowledge of and attitudes towards learning about the topic, often disengaging during learning (Corner et al., 2015; Kuthe et al., 2020). It has also been noted that this type of student tends to be more likely to feel psychologically distant from the problem: such disengaged students possess a low sense of agency and are less likely to participate in any social or environmental practices aimed at addressing the issue (Corner et al., 2015; Yilmaz & Can, 2019). Due to these factors, there has been a greater focus on incorporating stronger climate change education into science curricula and classroom learning (Hestness et al., 2019; Oliver & Adkins, 2020). Additionally, science researchers and educators have continued to explore ways of identifying effective climate change education strategies and programs (Monroe et al., 2019). The aim has been to improve student climate change literacy, attitudes, and awareness, so that they can understand the interdisciplinary connections and personal connection that the climate change issue has with their daily lives (Bunten & Dawson, 2014). This, in turn, helps to invoke students’ sense of agency and empowerment to make better-informed decisions and engage in social and environmental adaptation and mitigation practices (Trott, 2020).
Significance of This Study
This study seeks to fill the present gap in SSI-based STEAM research by designing and providing a program evaluation rubric that informs teachers about the key features for program development. As highlighted by Won et al. (2021), teachers face challenges in using SSI-based STEAM because they lack both the pedagogical knowledge and the understanding of the approach, as well as experience with planning and using the approach in their classrooms. To better guide teachers to develop quality SSI-based STEAM programs, it is necessary to provide them with program evaluation tools such as rubrics.
The rubric will allow teachers of any background and experience level to make evidence-based judgements about SSI-based STEAM programs rather than resorting to personal assumptions (Dickinson & Adams, 2017). It can also help them to articulate which parts of their programs align more easily and clearly with the criteria of the approach and with the 6E inquiry instructional model (Dickinson & Adams, 2017). Teachers will thus be able to collect evaluative information about the relevance, quality, and success of the programs in helping students to reach the short- and long-term learning goals targeted (Boyce, 2017; Martens, 2018a, b). The rubric also has the potential to be used in teacher professional development settings by providing well-defined criteria that can help teachers identify the weaknesses and strengths in their own practices and in the scope across which they teach certain science topics (Kind, 2019).
Purpose of This Study
The purpose of this study is to investigate how the ‘SSI-based STEAM Teacher Evaluation and Program Development’ (SSTEPD) rubric identifies the SSI-based STEAM approach in programs. Through this investigation, we aim to provide teachers with a program evaluation tool that can be used to help them plan, select, and assess the effectiveness of SSI-based STEAM programs and resources. This study will address the following research questions:
RQ1. How well does the SSTEPD rubric measure the SSI-based STEAM dimensions and principles in climate change programs?
RQ2. To what extent can the SSTEPD rubric be applied across different types of climate change programs?
Methodology
The SSTEPD rubric was developed following a multi-phase process adapted from design-based research (DBR) methodology (Brown, 1992). Drawing from the interconnected phases of the DBR method (Plomp, 2007), this study followed three phases: (1) preliminary research, (2) prototyping, and (3) assessment (Fig. 2). In Phase 1, preliminary research was conducted to identify the elements that needed to be incorporated into a defined rubric that can evaluate SSI-based STEAM programs. In Phase 2, the program evaluation rubric was developed based on the findings from the preliminary research and subjected to feedback and validation from an expert panel. Feedback from the expert panel was then used to revise the rubric. In Phase 3, the finalised program evaluation rubric was applied to actual programs to assess whether it could evaluate the specific criteria targeted. The information gathered was used to make further revisions and recommendations for future practice.
Preliminary Research Phase
The existing literature relevant to SSI-STEAM, scientific literacy, and inquiry-based learning was reviewed to identify key elements and goals consistent with the approach. From the literature review, several elements and key goals related to three main themes were identified: the SSI-based STEAM approach, the 6E instructional model, and the three visions of scientific literacy (Fig. 3). For example, it has been emphasised that SSI-based STEAM lessons need to encourage students to act on their own sense of social and scientific responsibility when addressing real-world issues (Mang et al., 2021). The use of inquiry-based instructions included the use of collaboration, hands-on activities, and posing questions (Mang et al., 2021). These elements identified from the literature aligned well with the three visions for scientific literacy. Vision I focuses on developing scientific knowledge, skills, and processes for understanding (Roberts, 2007). Vision II focuses on contextualising science so that a learner can understand how science is applied in real-world situations and human endeavours (Roberts & Bybee, 2014; Tan, 2016). Vision III focuses on integrating various perspectives of science from a socio-cultural and political point of view (Tan, 2016). The list of educational elements and goals helped define the key teaching and learning criteria and the evaluation categories.
Prototyping Phase
This phase comprises of two main components: (1) designing the rubric and (2) validating the items with a panel of experts.
Designing a Rubric
After the educational elements and goals were identified, they were used to generate measurable evaluation criteria. Each criterion had to define a key component of SSI-based STEAM and as such needed to reflect one or more of the key dimensions outlined in the SSI-based STEAM model (Appendix, Table 6). However, teachers need to plan and select activities that not only reflect the approach but also target certain learning opportunities embedded in the 6E inquiry model (Fig. 4). To assist teachers in establishing how to address the activities in the different inquiry stages in the 6E model and the general outcomes of SSI-based STEAM learning, we determined that the rubric needed seven evaluation categories: (1) learning outcomes, (2) engage, (3) explore, (4) explain, (5) elaborate, (6) evaluate, and (7) enact.
Learning opportunities targeted at different stages of the 6E inquiry model (Mang et al., 2021)
Once the evaluation criteria and categories were identified, various rubric guidelines were explored to determine the rubric format. To address concerns about time constraints and ease of use, a rubric checklist style was selected. This format allows teachers to determine the level of the performance without the complexity of detailed descriptors for each criterion found in traditional rubrics, which can be a lengthy process to complete (Enszer & Buckley, 2020). Teachers and educators rated the extent to which the program reflected the criteria on a 4-point Likert scale: great extent, (3) somewhat (2), very little (1), and not evident (0). This scale provided teachers with a score that could be tallied up at the end of the evaluation to provide an overall score total. For example, if the program facilitated opportunities for students to actively participate in developing local and global solutions to ‘a great extent’, then a score of 3 was given. The final score obtained gave teachers an indication of the overall quality of the SSI-based STEAM program.
Validation Process
After the first draft of the SSTEPD rubric had been created, it was validated by a panel of six experts from the fields of STEAM education, SSI education, and science education research. An online survey was administered to the panel to rate whether the rubric items targeted the constructs and were specific, measurable, attainable, realistic, and timely (SMART; Wilkerson & Haden, 2014). The ratings were used to establish the face validity and calculate the content validity ratio (CVR). Any item that did not achieve an CVR of 1 was deemed invalid and modified or removed. The ratings and additional written feedback guided three consultation sessions and new iterations of the rubric. The final rubric consisted of 37 items, each of which reflected one or more principles of the SSI-based STEAM model (Appendix, Table 6).
Assessment Phase
To determine that the rubric accurately represented and could evaluate the approach, two science educators, one from a school and one from a university, used the developed rubric to assess a newly developed SSI-based STEAM climate change program and five other existing climate change programs. The science educators independently rated and scored the programs using the SSTEPD rubric. Any differences in how scores were given or how criteria were interpreted were discussed until both educators reached an agreement. This process provided insights into how criteria were interpreted, and scores were allocated, and hence helped identify differences between the programs and how each program was addressing one or more SSI-based STEAM dimensions or principles. Afterwards, the researchers held joint sessions with all the science educators to discuss their ratings and determine how each expert had interpreted the rubric when providing their score. The climate change programs evaluated were developed by various parties including university researchers, government, and council initiatives, external education course providers, and schoolteachers (Table 2). Applying the SSTEPD rubric across a range of programs helped to establish whether it could be universally used to assess programs from other contexts. This evaluation allowed us to highlight the strengths and weaknesses of the rubric and make judgements on its suitability and practicality for implementation by teachers and researchers. This allowed for further recommendations to be made for future studies.
Results
The SSTEPD rubric was used to score the programs and the results were analysed to help answer two key questions regarding whether the rubric could (1) measure the SSI-based STEAM dimensions and principles and (2) be applied across different types of climate change programs. Two science educators independently evaluated and scored the program before engaging in discussions to compare the scores provided. The scoring was highly consistent between both the science educators and between the different programs. The kappa coefficient calculated showed the level of agreement was within the range 0.70–0.85, supporting a high level of inter-rater agreement (McHugh, 2012).
Measuring SSI-Based STEAM Dimensions and Principles
The final iteration of the SSTEPD rubric received strong agreement and positive feedback from the expert panel, who agreed that most of the evaluation criteria and categories would be able to measure SSI-based STEAM dimensions and principles clearly. To explore how well the SSTEPD rubric could achieve this goal in practice, the rubric was first applied to a newly developed SSI-based STEAM program on climate change. The educators found that the format of the SSTEPD rubric was user-friendly and allowed easy identification of evident or missing SSI-based STEAM components. However, there were a few issues that had an impact on each user’s interpretation of whether the program addressed a certain criterion. Firstly, it was unanimously agreed that certain terminology such as ‘social and scientific responsibility’ can be interpreted and defined differently depending on the user. It was recommended that a clear definition needed to be provided to users to ensure that there is a consensus on the meaning of the terms used. Secondly, there was a lack of explicit examples of what some activities may look like in practice. For example, it was unclear what type of activities reflected the ‘use of arts knowledge and practice to make connections to science and SSI’. Overall, the science educators still strongly agreed that the rubric was highly capable of measuring the SSI-based STEAM dimensions and principles in climate change programs.
Universality of the SSTEPD Rubric
To test the universality of the SSTEPD rubric, six different climate change programs developed by different program developers were evaluated using the rubric. These program developers included external education course providers Cura and Stile Education, Western Washington University, the Great Barrier Reef Marine Park Authority, and schoolteachers. It should be noted that while the programs were all on the same topic of climate change, they adopted different educational and inquiry approaches. As such, the study’s goal here was to determine what SSI-based STEAM components might already be present in these existing programs or where such components might be missing, thus offering a great opportunity for SSI-based STEAM inclusion in those existing programs. This process, in turn, would give a clear indication as to whether the SSTEPD rubric could consistently assess and score programs across various contexts. As shown in Table 3, the overall score was calculated and converted into percentage terms for scoring comparisons.
The SSI-Based STEAM Climate Change Project Group Program
This program demonstrated a high capacity to successfully address the criteria in the rubric and therefore received a score of 101 out of 111 (91%). This program was specifically designed to reflect the SSI-based STEAM approach, and as a result had a strong focus on using SSI to contextualise the science concepts and on creating wide ‘enact’ opportunities, for example, for students to critically evaluate the roles of values and identities in social practices. Thus, the program was able to score higher in many evaluation categories such as ‘engage’ and ‘explore’. However, the raters noticed that there was a lack of explicit inclusion of activities that facilitated the use of arts-related knowledge and practices to reinforce or make connections with science. This shortcoming had an impact on how well the program was able to address the principle of ‘interdisciplinary thinking and integrated practices’. As a result, this program scored poorly (M = 2.50) in the categories ‘explain’ and ‘elaborate’ (Table 4), where many of these types of activities were embedded.
The Abandon or Adapt? Program by Cura
This problem-based learning (PBL) STEM program scored 73 out of 111 (66%) and strongly demonstrated its ability to engage students in the inquiry process and thus develop their scientific knowledge and skills. The program includes many hands-on activities and collaborative learning devices to help students engage with their learning and offers several opportunities for students to reflect on and assess their own learning. However, it does not take advantage of several access points for integrating socio-scientific components such as exploring values, ethics, and morals. As a result, it received the lowest mean score (M = 1.33) in the ‘enact’ category. Furthermore, the program’s superficial application of the ‘interdisciplinary thinking and integrated practices’ concept, which is limited in this program to opportunities such as drawing, raised some concerns.
Earth Systems: Climate Change by Stile Education
This online module scored 62 out of 111 (56%) and focuses strongly on developing students’ scientific knowledge and understanding. It consists of many self-directed learning activities and promotes the use of student and teacher assessment to monitor student progress. This allowed the module to receive full scores in the ‘evaluation’ category (M = 3). Hands-on investigations and research activities were presented to engage students with the SSI topic and apply their scientific skills. However, at times, there were very limited opportunities present for students to pose questions or create hypotheses or predictions. The technology used was well integrated to help facilitate more blended learning and reinforce learning. Interdisciplinary thinking and integrated practices such as the use of the arts were adequately addressed in the module. However, there were missed opportunities to integrate socio-cultural components and perspectives or activities to facilitate the enactment of values and practices (M = 1). This really removed the affective learning experiences that would connect the learning more to students’ emotions. Overall, the program did not address many of the other SSI-based STEAM principles deeply enough and thus scored poorly across all categories (Table 4).
Climate Change: Connections and Solutions Program by Facing the Future
This interdisciplinary program developed by Western Washington University researchers received a high score of 109 out of 111 (98%). The program makes clear and authentic connections between climate change and students’ daily lives, and thus provides optimal opportunities for them to engage deeply with the SSI raised. For example, under this program, students engage in tasks that encourage them to consider the impact of their consumer behaviour on climate change. Furthermore, many of the program’s activities challenge students to use interdisciplinary perspectives to understand the program and develop solutions. Hence, the program received high mean scores (M = 3) in most of the evaluation categories. However, the program could offer greater benefits to students if it provided more opportunities for more diverse technologies, designs, and practices to help them understand and explain concepts. Additionally, this program does not promote more in-depth discussion about different cultural and personal values. As shown in Table 4, these factors lowered the program’s mean scores for the evaluation categories ‘explain’ (M = 2.80) and ‘enact’ (M = 2.83).
The Reef Guardians: Climate Change Program by the Great Barrier Reef Marine Park Authority
This council program scored 85 out of 111 (77%). The program follows the 5E inquiry model and provides many hands-on activities such as investigations and group tasks. This facilitates quality opportunities for students to collaborate with their peers. However, the program follows a teacher-guided inquiry approach, which limits opportunities for learners to engage in more student-driven activities to understand the problem more openly. This made it challenging to see how the program could effectively address SSI-based STEAM principles, and thus lowered the program’s scores across all evaluation categories in our study (Table 3). The program did put forward perspectives from various other dimensions, such as politics and economics. However, it neglected to introduce students to perspectives from dimensions such as ethics, culture, and values. As a result, it appeared that this program could not facilitate opportunities for students to apply values and social practices; hence, it scored very poorly in the ‘enact’ category (M = 0.83).
Program Developed by Schoolteachers
This teacher-developed program performed the worst of all the programs, with a score of 47 out of 111 (42%). It was difficult to determine what inquiry model had been adopted in the program as many of its activities were teacher directed. There was an over-reliance on the use of research tasks, lectures, and worksheets, which hindered opportunities for students to engage actively as learners or in student-driven activities. The content focused strongly on the science content, such as greenhouse gases and the carbon cycle, reinforced by some opportunities for hands-on investigations. However, it was noted that the tasks only superficially linked the science concepts to climate change as an SSI. Limited opportunities were provided to students to engage in strong discourse about the real-world problem of climate change or to develop and apply wider perspectives from areas such as culture and ethics. This did not allow the program to adequately address any of the important principles of SSI-based STEAM—‘enacted values and practices’, ‘interdisciplinary learning and integrated practices’, ‘authentic context and activities’, and ‘affective learning’, for example. This resulted in low mean scores (M = 1.80) for this program across all evaluation categories.
Discussion
The intent of this SSTEPD rubric was to help guide teachers and educators in identifying features of SSI-based STEAM, measuring the quality of their programs, and developing their own programs. Moreover, teachers could use the tool to ensure that there was alignment between the curriculum and goals of SSI-based STEAM learning. By consulting a panel of experts from relevant fields, the study was able to produce a rubric that reflected (1) the four key principles of SSI-based STEAM, (2) the 6E inquiry model, and (3) the three visions of scientific literacy. To ensure the SSTEPD rubric can assess the activities and outputs targeted, the tool was applied to various climate change programs. The SSTEPD rubric demonstrated the ability to rate the principles and dimensions reliably and consistently in various climate change programs. In doing so, the rubric was also able to highlight gaps and identify areas in the programs where there were opportunities to include SSI-based STEAM learning.
However, we noted that in some items, terms, such as ‘social action’, were ambiguous, so their interpretation is dependent on the rubric user. This leaves room for discrepancies to arise and thus has an impact on how certain programs may be scored between different rubric users. These discrepancies may become more apparent between different stakeholders who may assign their own values and goals to the programs (Dickinson & Adams, 2017). Another area of concern about the SSTEPD rubric relates to some items’ focus on assessing whether a lesson could facilitate opportunities for student experiences in areas such as ‘social and scientific responsibility’. Based solely on the wording of the criteria and with no example of how a concept like this can be represented through a practical activity, it can be difficult for teachers to know what to look for in the programs and how to assign a score (Dickinson & Adams, 2017). It is important to clearly define how these principles or dimensions can be manifested as practical activities (Allen & Tanner, 2006).
The checklist format of the SSTEPD rubric engages teachers in a process of rating and scoring the evaluation criteria, and thus captures qualitative information about the quality of the program being evaluated. Additionally, the checklist style helps teachers to avoid the potential problem of describing and justifying the grade given to work that may fall between performance descriptors (Enszer & Buckley, 2020). Teachers can then easily and quickly gather evidence about how the program reflects SSI-based STEAM and how well it meets the criterion, and they can use it to guide self-assessment (Bharuthram & Patel, 2017). However, the format does not allow the option for the teacher to note written or verbal feedback during their evaluation. Supplementing the SSTEPD rubric with an additional column for annotations or notes about evidence for scoring or suggestions for modification could further promote teacher reflection and a narrative for discussion between colleagues (Dawson, 2017).
Despite the differences in the activities, frameworks, and pedagogical approaches adopted in the programs, the rubric demonstrated its ability to consistently give scores across the categories and to highlight certain trends. This is because the dimensions and principles of SSI-based STEAM (e.g., authentic context and tasks) are deployed in many educational approaches including STEM. Therefore, we believe that the differences between the programs do not affect or detract from the capabilities of the SSTEPD rubric. In fact, the SSTEPD rubric was designed to be specific in targeting the evaluation contexts, which allows the tool to be applied across different contexts and types of programs (Martens, 2018a, b). For example, five out of six of the programs showed a weakness in addressing the ‘enact’ category, failing to identify sufficient connections with socio-scientific dimensions such as values and social practices. As a result, students were not provided with adequate opportunities to put into practice their understanding of the roles of different values and practices in decisions made to combat climate change. Another trend observed across the programs was the strong focus on assessment. This is likely due to the importance that is placed on student performance and gathering evidence of students’ achievements relevant to desired outcomes, as well as other concerns such as teacher accountability (Bell & Cowie, 2001).
Overall, the SSTEPD rubric was able to provide an insight into the quality of the programs examined in relation to SSI-based STEAM learning. The Climate Change: Connections and Solutions (Facing the Future) program from Western Washington University was explicitly developed as an interdisciplinary program that effectively made strong connections between different dimensions and thus scored highly on the rubric. The other programs developed by external education course providers scored relatively well but were very weak in terms of reflecting the dimensions of ‘enacting values and practices’, ‘affective learning’, and ‘interdisciplinary thinking and integrated practices’. The program developed by schoolteachers showed the lowest scores across all dimensions. The low scores can be attributed to the lack of cohesion between program activities, the failure to consider how various dimensions interconnect with each other, and how this could facilitate a more connected learning experience for students.
Benefits of the Rubric
Most programs are created with the intention of assisting teachers to educate students effectively and achieve targeted educational objectives (Centre for Education Statistics and Evaluation, 2015). Rubrics are a great tool for setting teachers’ expectations when they assess the delivery and quality of the activities they plan in their programs so that they can achieve their targeted outcomes (Allen & Tanner, 2006). The SSTEPD rubric breaks down the specific features or instructional strategies that are being assessed and draws teachers’ focus to the strengths, weaknesses, and gaps in their own programs, thus guiding program development. Not only can teachers evaluate their own program using a rubric, but also the rubric can aid them in establishing the merit of other existing programs based on standards and evidence that allow inter-program comparisons and conclusions to be made (Dickinson & Adams, 2017). Furthermore, the use of a rubric has merit for outcome evaluation, in that it can be used to determine the impact the program has on students’ learning outcomes and hence determine whether it is suitable for the target audience (Centre for Education Statistics and Evaluation, 2015). From this process, teachers can more effectively assess the quality of a program and can judge whether it has value for teaching and learning (Allen & Tanner, 2006).
The SSTEPD rubric could also provide consistent and explicit expectations between teachers within the same faculty or across multiple disciplines (Timmerman, 2011). This, in turn, could help faculties to plan more holistic assessments that will engage students in both the outcomes and the processes of learning (Timmerman, 2011). In these cases, where more than one rubric user is involved, the SSTEPD rubric would facilitate clearer communications about the expectations and standards that constitute quality programs and how these expectations are reflected in classroom learning (Dickinson & Adams, 2017). Through a communication process, teachers can then start to form a better idea about what constitutes quality SSI-based STEAM programs. Moreover, the evaluation data collected will streamline the cycle of evaluating, analysing, receiving feedback, and revising the programs and related materials (Dickinson & Adams, 2017; Wilkerson & Haden, 2014). This facilitates teachers’ deeper involvement and engagement with their teaching tools, enabling them to understand and reflect on their own practices (Allen & Tanner, 2006; Kind, 2019). Overall, the program evaluation tool is a valuable resource for teachers currently implementing or wishing to implement SSI-based STEAM, who, due to a lack of guidance, self-efficacy, or knowledge, find it difficult to make science more meaningful to their students by engaging them in developing a greater sense of responsibility about social scientific issues (Martens, 2018a, b; Won et al., 2021).
Conclusions and Future Studies
The SSTEPD rubric is a valuable tool that provides a tangible way for teachers to assess, plan, and implement SSI-based STEAM and that has several implications for teaching and learning. Firstly, the rubric highlights for teachers the criteria that they need to look for when developing and evaluating their own and existing programs. Secondly, it provides a benchmark for teachers to communicate standards and expectations among and across faculty members, and to discuss the strengths and weaknesses of activities and programs implemented (Dickinson & Adams, 2017). The rubric can also be used to promote negotiations among teachers about what elements in their program need to be revised (Martens, 2018a, b). This can encourage teachers to reflect on and improve their own teaching practices (Allen & Tanner, 2006; Martens, 2018a, b). Lastly, the rubric has the potential to guide the planning and design of new programs and activities (Cornman et al., 2013).
Although the SSTEPD rubric was developed based on the expertise and feedback of experts and researchers in the field, it still lacks critical input from teachers. As a result, it is difficult to determine definitively whether it has practical applications in the classroom and is user-friendly and whether teachers have the skills and experience to utilise the rubric effectively (Martens, 2018a, b). Future studies need to involve teachers in any rubric revisions and explore how the SSTEPD rubric works in the classroom setting. This will provide valuable insights into what factors may have an impact on the use of the tool, such as teaching experience with using rubrics, evaluation practices, and time constraints. Additionally, it may reveal gaps in how professional teacher workshops may use the rubric to expand teachers’ understanding and acceptance of the SSI-based STEAM approach, and their willingness to implement it in their classrooms.
Change history
23 January 2024
A Correction to this paper has been published: https://doi.org/10.1007/s11165-024-10154-9
References
Abed, O. H. (2016). Drama-based science teaching and its effect on students’ understanding of scientific concepts and their attitudes towards science learning. International Education Studies, 9(10), 163–173.
Alcaraz-Dominguez, S., & Barajas, M. (2021). Conceptualization of socioscientific issues in educational practice from a review of research in science education. International Journal of Information and Education Technology, 11(6), 297–203. https://doi.org/10.18178/ijiet.2021.11.6.1526
Allen, D., & Tanner, K. (2006). Rubrics: Tools for making learning goals and evaluation criteria explicit for both teachers and learners. CBE Life Sciences Education, 5(3), 197–203. https://doi.org/10.1187/cbe.06-06-0168
Australian Bureau of Statistics. (2020). Droughts, fires, cyclones, hailstorms and a pandemic the march quarter 2020. Retrieved 13 Apr 2023 from https://www.abs.gov.au/articles/droughts-fires-cyclones-hailstorms-and-pandemic-march-quarter-2020
Ayre, C., & Scally, A. J. (2014). Critical values for Lawshe’s content validity ratio: Revisiting the original methods of calculation. Measurement and Evaluation in Counseling and Development, 47(1), 79–86. https://doi.org/10.1177/0748175613513808
Baek, S., Shin, H., & Kim, C. J. (2022). Development of a climate change SSIBL-STEAM program aligned to the national curriculum for SSI elementary school in Korea. Asia-Pacific Science Education, 8, 109–148. https://doi.org/10.1163/23641177-bja10047
Bell, B., & Cowie, B. (2001). The characteristics of formative assessment in science education. Science Education (Salem, Mass.), 85(5), 536–553. https://doi.org/10.1002/sce.1022
Bharuthram, S., & Patel, M. (2017). Co-constructing a rubric checklist with first year university students: A self-assessment tool. Apples: Journal of Applied Language Studies, 11(4), 35–55. https://doi.org/10.17011/apples/urn.201708073430
Boice, K. L., Jackson, J. R., Alemdar, M., Rao, A. E., Grossman, S., & Usselman, M. (2021). Supporting teachers on their STEAM journey: A collaborative STEAM teacher training program. Education Sciences, 11(3), 105. https://doi.org/10.3390/educsci11030105
Boyce, A. S. (2017). Lessons learned using a values-engaged approach to attend to culture, diversity, and equity in a STEM program evaluation. Evaluation and Program Planning, 64, 33–43.
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141–178. https://doi.org/10.1207/s15327809jls0202_2
Bunten, R., & Dawson, V. (2014). Teaching climate change science in senior secondary school. Australian Science Teachers Association, 60(1), 10.
Bybee, R. W., Taylor J. A., Gardner, A., Scotter, P. V., Powell, J. C., Westbrook, A., & Landes, N. (2006). The BSCS 5E instructional model: Origins and effectiveness. Retrieved 2 Aug 2022 from https://bscs.org/sites/default/files/_media/about/downloads/BSCS_5E_Full_Report.Pdf
Centre for Education Statistics and Evaluation. (2015). 5 essentials for effective evaluation. Retrieved 18 Aug 2022 from https://www.cese.nsw.gov.au//images/stories/PDF/5_Essentials_for_Effective_Evaluation_FA_AA.pdf
Choi, S. Y., Won, A. R., Chu, H. E., Cha, H. J., Shin, H., & Kim, C. H. (2021). The impacts of a climate change SSI-STEAM program on junior high school students’ climate literacy. Asia-Pacific Science Education, 7(1), 96–133. https://doi.org/10.1163/23641177-bja10019
Chu, H. E., Martin, S. N., & Park, J. (2019). A theoretical framework for developing an intercultural STEAM program for Australian and Korean students to enhance science teaching and learning. International Journal of Science and Mathematics Education, 17(7), 1251–1266. https://doi.org/10.1007/s10763-018-9922-y
Clément, P. (2012). Values in science and in science education. Science & Technology Education for Development, Citizenship and Social Justice, 1(1), 26.
Cook, K., Bush, S., Cox, R., & Edelen, D. (2020). Development of elementary teachers’ science, technology, engineering, arts, and mathematics planning practices. School Science and Mathematics, 120(4), 197–208. https://doi.org/10.1111/ssm.12400
Cornman, S., Holliday, G., & Tersoff, G. (2013). Developing an evaluation rubric for hands-on science education activities. Melbourne: Worcester Polytechnic Institute. Retrieved on 12 Feb 2021 from https://digital.wpi.edu/concern/student_works/g158bh68n?locale=en
Corner, A., Roberts, O., Chiari, S., Völler, S., Mayrhuber, E., Mandl, S., & Monson, K. (2015). How do young people engage with climate change? The role of knowledge, values, message framing, and trusted communicators. Wires Climate Change, 6(5), 523–534. https://doi.org/10.1002/wcc.353
Cura. (2021). Abandon or adapt. Retrieved 15 Mar 2020 from https://www.curaeducation.com/all-units/abandon-or-adapt
Dawson, P. (2017). Assessment rubrics: Towards clearer and more replicable design, research and practice. Assessment & Evaluation in Higher Education, 42(3), 347–360. https://doi.org/10.1080/02602938.2015.1111294
Dickinson, P., & Adams, J. (2017). Values in evaluation – The use of rubrics. Evaluation and Program Planning, 65, 113–116. https://doi.org/10.1016/j.evalprogplan.2017.07.005
Ekborg, M., Ideland, M., & Malmberg, C. (2009). Science for life – A conceptual framework for construction and analysis of socio-scientific cases. Nordina, 5(1), 35–46. https://doi.org/10.5617/nordina.277
Enszer, J. A., & Buckley, J. M. (2020). Algorithm for consistent grading in an introduction to engineering course. 2020 ASEE Virtual Annual Conference Content Access Proceedings. https://doi.org/10.18260/1-2--34100
Facing the Future, Western Washington University, “Climate Change: Connections and Solutions - Recommended for Grades 9–12” (2013). Facing the future. 7. Retrieved 2 Jul 2020 from https://cedar.wwu.edu/ee_ftf/7
Filkov, A. I., Ngo, T., Matthews, S., Telfer, S., & Penman, T. D. (2020). Impact of Australia’s catastrophic 2019/20 bushfire season on communities and environment. Retrospective analysis and current trends. Journal of Safety Science and Resilience, 1(1), 44–56. https://doi.org/10.1016/j.jnlssr.2020.06.009
French, K. R. (2020). Student teaching and urban educator aptness: The significance of similar sociocultural scenarios. Education and Urban Society, 52(4), 511–533. https://doi.org/10.1177/0013124519877162
Gao, L., Mun, K., & Kim, S. W. (2019). Using socioscientific issues to enhance students’ emotional competence. Research in Science Education, 51(2), 935–956. https://doi.org/10.1007/s11165-019-09873-1
Goodrum, D., & Rennie, L. (2007). Australian school science education national action plan 2008 – 2012. Department of Education, Science and Training, Commonwealth of Australia. Retrieved from https://files.eric.ed.gov/fulltext/ED511041.pdf
Great Barrier Reef Marine Park Authority. (2016). Climate change: Year 10 Australian science curriculum focus. Retrieved 17 May 2020 from https://elibrary.gbrmpa.gov.au/jspui/handle/11017/2995
Herro, D., & Quigley, C. (2016). Exploring teachers’ perceptions of STEAM teaching through professional development: implications for teacher educators. Professional development in Education, 43(3), 416–438. https://doi.org/10.1080/19415257.2016.1205507
Herro, D., Quigley, C., & Cian, H. (2019). The challenges of STEAM instruction. Lessons from the field. Action in Teacher Education, 41(2), 172–190. https://doi.org/10.1080/01626620.2018.1551159
Herro, D., Quigley, C., Andrews, J., & Delacruz, G. (2017). Co-measure: Developing an assessment for student collaboration in STEAM activities. International Journal of STEM Education, 4(1), 2–12. https://doi.org/10.1186/s40594-017-0094-z
Hestness, E., McGinnis, J. R., & Breslyn, W. (2019). Examining the relationship between middle school students’ sociocultural participation and their ideas about climate change. Environmental Education Research, 25(6), 912–924. https://doi.org/10.1080/13504622.2016.1266303
Hong, O. (2021). STEM/STEAM education research in South Korea. In STEM education from Asia: Trends and perspectives (pp 211–227). https://doi.org/10.4324/9781003099888-11
Intergovernmental Panel on Climate Change. (2018). Sixth assessment report. Retrieved 22 Mar 2023 from https://www.ipcc.ch/assessment-report/ar6/
Kind, V. (2019). Development of evidence-based, student-learning oriented rubrics for pre-service science teachers’ pedagogical content knowledge. International Journal of Science Education, 41(7), 911–943. https://doi.org/10.1080/09500693.2017.1311049
Klosterman, M. L., & Sadler, T. D. (2010). Multi-level assessment of scientific content knowledge gains associated with socioscientific issues-based instruction. International Journal of Science Education, 32(8), 1017–1043. https://doi.org/10.1080/09500690902894512
Kuthe, A., Körfgen, A., Stötter, J., & Keller, L. (2020). Strengthening their climate change literacy: A case study addressing the weaknesses in young people’s climate change awareness. Applied Environmental Education & Communication, 19(4), 375–388. https://doi.org/10.1080/1533015X.2019.1597661
Lee, E. A., & Brown, M. J. (2018). Connecting inquiry and values in science education an approach based on John Dewey’s philosophy. Science & Education, 27(1–2), 63–79. https://doi.org/10.1007/s11191-017-9952-9
Lee, H., Yoo, J., Choi, K., Kim, S., Krajcik, J., Herman, B. C., & Zeidler, D. L. (2013). Socioscientific issues as a vehicle for promoting character and values for global citizens. International Journal of Science Education, 35(12), 2079–2113. https://doi.org/10.1080/09500693.2012.749546
Lin, C., Huang, J., & Lin, R. (2021). From steam to cheer: A case study of design education development in Taiwan. Education Sciences, 11(4), 171. https://doi.org/10.3390/educsci11040171
Mang, H., & Chu, H-E. (2020). SSI-based STEAM program for years 9 and 10: Climate Change. Unpublished Program.
Mang, H. M. A., Chu, H. E., Martin, S. N., & Kim, C. (2021). An SSI-based STEAM approach to developing science programs. Asia-Pacific Science Education, 7(2), 549–585. https://doi.org/10.1163/23641177-bja10036
Martens, K. S. R. (2018a). Rubrics in program evaluation. Evaluation Journal of Australasia, 18(1), 21–44. https://doi.org/10.1177/1035719X17753961
Martens, K. S. R. (2018b). How program evaluators use and learn to use rubrics to make evaluative reasoning explicit. Evaluation and Program Planning, 69, 25–32. https://doi.org/10.1016/j.evalprogplan.2018.03.006
McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochem Medica, 22(3), 276–282.
Monroe, M. C., Plate, R. R., Oxarart, A., Bowers, A., & Chaves, W. A. (2019). Identifying effective climate change education strategies: A systematic review of the research. Environmental Education Research, 25(6), 791–812. https://doi.org/10.1080/13504622.2017.1360842
NSW State of the Environment. (2021). Fire. Retrieved 17 Apr 2021 from https://www.soe.epa.nsw.gov.au/all-themes/land/fire
Oliver, M. C., & Adkins, M. J. (2020). “Hot-headed” students? Scientific literacy, perceptions and awareness of climate change in 15-year olds across 54 countries. Energy Research & Social Science, 70, 101641. https://doi.org/10.1016/j.erss.2020.101641
Park, Y. S., & Park, J. H. (2020). Exploring the explicit teaching strategies in STEAM program of climate change. Asia-Pacific Science Education, 6(1), 116–151. https://doi.org/10.1163/23641177-BJA00002
Plomp, T. (2007). Educational design-based research: An introduction. In T. Plomp & N. Nieveen, (Eds) Educational design-based research: Part A (p.10–51). Netherlands Institute for Curriculum Development (SLO).
Quigley, C. F., & Herro, D. (2016). “Finding the joy in the unknown”: Implementation of STEAM teaching practices in middle school science and math classrooms. Journal of Science Education and Technology, 25(3), 410–426. https://doi.org/10.1007/s10956-016-9602-z
Reis. P. (2014). Promoting students’ collective socio-scientific activism: Teachers’ perspectives. In Activist science and technology education (pp. 547–574). Springer Netherlands. https://doi.org/10.1007/978-94-007-4360-1_31
Roberts, D. A. (2007). Scientific Literacy/Science Literacy. In S. K. Abell & N. G. Lederman (Eds.), Handbook of research on science education (pp. 729–781). Routledge.
Roberts, D., & Bybee, R. (2014). Scientific literacy, science literacy and science education. In N. G. Lederman & S. K. Abell (Eds.), Handbook of research on science education (pp. 545–558). Routledge.
Rodrigues-Silva, J., & Alsina, A. (2022). Effects of a practical teacher-training program on STEAM activity planning. Revista Tempos e Espaços em Educação, 15(34), e17993. https://doi.org/10.20952/revtee.v15i34.17993
Sadler, T. D. (2011). Situating socio-scientific issues in classrooms as a means of achieving goals of science education. In T. D. Sadler (Ed.), Socio-scientific issues in the classroom (pp. 1–9). Springer, Netherlands. https://doi.org/10.1007/978-94-007-1159-4_1
Sadler, T. D., Chambers, F. W., & Zeilder, D. L. (2004). Student conceptualizations of the nature of science in response to a socioscientific issue. International Journal of Science Education, 26(4), 378–409. https://doi.org/10.1080/0950069032000119456
Sjöström, J., Frerichs, N., Zuin, V. G., & Eilks, I. (2017). Use of the concept of Bildung in the international science education literature, its potential, and implications for teaching and learning. Studies in Science Education, 53(2), 165–192. https://doi.org/10.1080/03057267.2017.1384649
Stile Education. (2021). Earth’s systems: climate change. Retrieved May 17, 2020, from www.stileeducation.com
Tan, P. (2016). Science education: Defining the scientifically literate person. SFU educational Review, 9, 1–8. https://doi.org/10.21810/sfuer.v9i.307
Timmerman, B. E. C. (2011). Development of a ‘universal’ rubric for assessing undergraduates’ scientific reasoning skills using scientific writing. Assessment & Evaluation in Higher Education, 36(5), 509–547. https://doi.org/10.1080/02602930903540991
Trott, C. D. (2020). Children’s constructive climate change engagement: Empowering awareness, agency, and action. Environmental Education Research, 26(4), 532–554. https://doi.org/10.1080/13504622.2019.1675594
Wilkerson, S. B., & Haden, C. M. (2014). Effective practices for evaluating STEM out-of-school time programs. Afterschool Matters, 19, 10–19.
Won, A. R., Choi, S. Y., Chu, H. E., Cha, H. J., Shin, H., & Kim, C. J. (2021). A teacher’s practical knowledge in an SSI-STEAM program dealing with climate change. Asia-Pacific Science Education, 7(1), 134–172. https://doi.org/10.1163/23641177-bja10023
Yakman, G., & Lee, H. (2012). Exploring the exemplary steam education in the U.S. as a practical educational framework for Korea. Journal of The Korean Association For Science Education, 32(6), 1072–1086. https://doi.org/10.14697/jkase.2012.32.6.1072
Yakman, G. (2008). STE@M education: An overview of creating a model of integrative education. Retrieved 19 Apr 2008 from https://www.iteea.org/File.aspx?id=86752&v=75ab076a
Yilmaz, V., & Can, Y. (2019). Impact of knowledge, concern and awareness about global warming and global climatic change on environmental behavior. Environment, Development and Sustainability, 22(7), 6245–6260. https://doi.org/10.1007/s10668-019-00475-5
Zeidler, D. L. (2016). STEM education: A deficit framework for the twenty first century? A sociocultural socioscientific response. Cultural Studies of Science Education, 11(1), 11–26. https://doi.org/10.1007/s11422-014-9578-z
Zeidler, D. L., & Nichols, B. H. (2009). Socioscientific issues: Theory and practice. Journal of Elementary Science Education, 21(2), 49–58. https://doi.org/10.1007/BF03173684
Funding
Open Access funding enabled and organised by CAUL and its member institutions. This work was supported by the Australian Government’s Department of Foreign Affairs and Trade (Australia-Korea Foundation, AKF Grant 2018040).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Mang, H.M.A., Chu, HE., Martin, S.N. et al. Developing an Evaluation Rubric for Planning and Assessing SSI-Based STEAM Programs in Science Classrooms. Res Sci Educ 53, 1119–1144 (2023). https://doi.org/10.1007/s11165-023-10123-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11165-023-10123-8