Introduction

Representation of under-represented minority groups (URMs) in science and engineering professions has historically been low and is often not reflective of the cultures and people that make up many communities across the United States and around the world (Agrawal et al. 2016; Allen-Ramdial and Campbell 2014; Estrada et al. 2016; Hall et al. 2017). For the purposes of this discussion, URMs include the following: biracial individuals and people of color (BIPOCs), women, members of the lesbian, gay, bisexual, transgender and queer (LGBTQ) community, and individuals with disabilities. A lack of representation in STEM fields presents significant ethical and societal challenges not only within these fields, but also for the larger economy and educational systems that support STEM professions (Hong and Page 2004; Grossman and Porche 2013). As broad consumer use of technology increases, and findings from scientific research and development become more publically available, the need for equitable dissemination of scientific information and transparent communication also increases (Hodson 2003). Arguably, one of the most feasible ways to ensure that all communities can fully participate in these innovations is to achieve proportional representation of community members in positions of power and influence within STEM fields.

The question then shifts to how best to accomplish this goal of enhanced representation? Specifically, in the context of education, at what point in the academic pipeline should students from URMs be targeted to most effectively increase long-term representation of these groups in STEM professions? Numerous studies have evaluated interventions aimed at increasing URM representation in higher education STEM disciplines (e.g., by promoting lower attrition rates in undergraduate science and engineering majors) (Griffith 2010; Habig et al. 2020; Olszewski-Kubilius et al. 2016; Palmer et al. 2018; Romine and Sadler 2016). While the emphasis on this more academically advanced population is a logical and important one, developmental psychology introduces a compelling argument for focusing on early science educational intervention, namely interventions targeting students aged 6 through 10 years old (Bailey 2002; Daugherty et al. 2014; Ejiwale 2013). There exists a rich body of literature on critical periods of development, and the role these stages play in the cognitive growth of an individual (Abello-Contesse 2008; Graf et al. 2021; Maggi et al. 2010). Even without universal agreement in the field, many psychologists acknowledge that these critical periods are important in shaping development to some extent, though the degree to which this influence is long-lasting is still debated. Therefore, making elementary school students the target population of STEM educational interventions presents an opportunity for early exposure to the language, disciplinary content and even the methodology and inner-workings of STEM professions (with these professions including, but not being limited to: research in academia and industry, biotechnology and bioengineering, pharmacology and medicine, engineering and design, science communication, and education). In addition, such early exposure would help to foster enhanced science literacy among a younger student population, ensuring that students are better prepared to understand, critically interpret, and integrate scientific language in an interdisciplinary manner in the more advanced stages of their academic career (Eshach 2006; Roberts and Bybee 2014).

It is important to note that the organization, nature, and goal of interventions targeting elementary school students will almost certainly be/are distinct from their higher education counterparts, and numerous models have been proposed to help guide educators and administrators in their approach to integrating STEM education into mainstream curricula (Chiu et al. 2015; Daugherty et al. 2014; Ejiwale 2013; English 2017; Faber et al. 2013). As with the educational program evaluated for this paper, the overarching goal in providing younger students with supplementary, informal science education may be to simply introduce students to scientists and professionals in STEM that represent them and/or come from a similar background, while also educating students on the core vocabulary, concepts, and theories associated with different STEM fields. Given that long-term retention and comprehension of scientific knowledge are limited for younger students (particularly when a more traditional, non-active learning-based approach to science education is taken), the goal of increasing enthusiasm for and interest in STEM is a realistic one. If achieved, it will ultimately augment the proportion of URM students that seek further science education of their own volition (Doran and Ngoi 1979; Upadhyay and DeFranco 2008).

The following paper evaluates the efficacy of an elementary school STEM educational program, known as the Young Scientists Program (YSP), which serves over 1400 elementary school students in seven Title I schools within the Los Angeles Unified School District (LAUSD), the second-largest public school district in the United States. The specific aim of this project was to assess whether targeting a younger student population using a hands-on, inquiry-based, and NGSS-aligned 18-week STEM curriculum would be effective in promoting a greater sense of self-efficacy within and/or heightened interest in science over the course of a single academic year. Future iterations of this quasi-experimental, pre-/post-evaluation protocol will be necessary to determine if the YSP is successful in ultimately encouraging students to see themselves as potential future scientists, regardless of their socioeconomic status and cultural background.

Methods

Young Scientists Program (YSP) structure, staff, and curriculum

The YSP is a supplementary STEM educational initiative designed to provide elementary school students (second–fifth grades, including Special Education students) with hands-on, inquiry-based science lessons and activities, with the long-term goal of encouraging students to see themselves as potential future scientists regardless of their academic and cultural background. The program has thrived under the umbrella administration of the Joint Educational Project (JEP) at the University of Southern California (USC) since 2005, employing over 150 undergraduate and graduate students interested in pursuing careers in science education and communication, with most these students themselves identifying as members of a URM (“Joint Educational Project USC Dana and David Dornsife College of Letters, Arts and Sciences”). While the YSP has expanded to serve more Title I elementary schools within LAUSD (the program currently serves seven different schools in the south-central Los Angeles area), the overall approach to curriculum development and pedagogical training has remained consistent, with credentialed educators helping to provide annual feedback on existing lessons, as well as participating in YSP-organized professional development seminars intended to support both pre-service learning educators and fully credentialed teachers in the participating LAUSD schools. In this way, the YSP serves as a unique educational programmatic model for addressing the unique academic, cultural, and financial needs of a diverse urban community in the United States, while also forming partnerships between higher educational institutions (in this case, USC) and the surrounding community.

For this specific study, 20 USC undergraduate and 4 graduate student teaching assistants underwent training in the early fall of 2018 to become familiar with the academic curriculum for their assigned grade level, with each teaching assistant being responsible for instructing anywhere from one to four separate classes at a designated school site and within their assigned grade level. Of these 24 teaching assistants, 22 were identified as female and 20 were identified as coming from a biracial, indigenous, or other ethnic/racial minority background. It is worth noting that one of the undergraduate teaching assistants was a previous participant in the YSP during their elementary school education and thus had personal experience and a unique perspective to lend to their teaching and other responsibilities in the program. In addition to curricular content training, the teaching assistant staff received guidance from specialists in the USC Rossier School of Education on how to address a wide array of individual student learning needs, and how to incorporate the Scientific Method into an active learning approach for teaching the YSP curriculum. All YSP lessons were aligned to appropriate grade-level Next-Generation Science Standards (NGSS) and were reviewed by credentialed teachers and educational staff to validate the appropriateness and inclusivity of the content and language (Achieve, Inc. 2013). Elementary school students participating in the program were taught by one teaching assistant for the duration of the academic year (over the course of an 18-week curriculum), and lessons were grouped into modules based on disciplinary content and the core vocabulary/concepts introduced to students through those lessons. Due to the high percentage of English Language Learners (ELL) in participating classrooms, lesson plans and other curricular materials were translated into Spanish, and teaching assistants in dual-language classrooms provide limited instruction in Spanish.

Student participants

Elementary school (2nd–5th grade) students attending the following schools were participants in YSP for the 2018–2019 Academic Year: 32nd Street USC Performing Arts Magnet School, Mack Elementary School, Foshay Learning Center, Lenicia B. Weemes Elementary School, Vermont Avenue Elementary School, Norwood Street Elementary School and the Dr. Theodore T. Alexander California Science Center School. Individualized gender, age, and demographic data were not collected on these students, and the public demographic reports for each school are summarized in Table 1 [District Profile: Los Angeles Unified (CA Department of Education) 2021]. Prior to the start of the program, over 1000 students completed the pre-program Science Interest Surveys and “Draw A Scientist” Tests (DASTs). Upon completion of the program, approximately two-thirds of the students from the same original cohort completed the post-program versions of these surveys/tests (with most non-responses resulting from the absence from school on the day the post-program surveys were administered). Surveys and tests were administered by trained undergraduate and graduate student teaching assistants within YSP, with assistance from classroom teachers and teaching aids.

Table 1 School demographics
Full size table

School district demographics

The LAUSD is the second-largest public school district in the United States and consists of a combination of traditional public schools and non-traditional (charter/magnet) schools [District Profile: Los Angeles Unified (CA Department of Education) 2021]. A total of 2923 students attended the seven elementary schools that received YSP instruction for the 2018–2019 academic year, though not all of these students participated in the YSP due to one (or more) of the following reasons: the classroom teacher did not opt-in to the program and/or the student’s grade level was not yet eligible to receive YSP instruction. Out of this total (n = 2923), 87.71% of students received free-or-reduced lunches, and 36.16% of students were ELLs. The majority (98.58%) of these ELL students are native Spanish speakers. The gender distribution across all elementary school students at these seven schools was nearly even, with female students making up 51.16% of the total, and male students making up the remaining 48.83%. The ethnicity breakdown of these students was as follows: 82.73% Latinx, 12.62% African American, and 0.1% American Indian or Alaskan Native. All seven schools are situated in the south-central Los Angeles area, which, according to the latest federal government census data, is a low-income community with more than twice the rate of poverty as the statewide average (US Census Bureau 2019).

Survey instruments

Science interest surveys

To gain a better understanding of student preconceptions and initial reactions to science, as well as measure the potential change in these perceptions/reactions over the course of participation in the YSP, a general Science Interest Survey was created by the YSP veteran research assistant team. The survey includes six questions with mixed question formats (some questions require a circled response from a set of options, while others require the student to provide a short, written response), with all questions being adapted from existing, validated survey instruments, including the Rasch-validated Science Interest Survey as well as the STEM Career Interest Survey (Kier et al. 2013; Lamb et al. 2011). Questions were selected and modified to ensure the language was easy to comprehend for all students regardless of their grade level, individual reading ability and English language proficiency. The survey was administered immediately prior to the “Draw A Scientist Test” (DAST) at both time points (pre- and post-program) while students participated in their first YSP-facilitated lesson (for the pre-program survey) and their last YSP-facilitated lesson (for the post-program survey). A version of the survey was also adapted to accommodate students with moderate-to-significant physical or learning disabilities, which eliminates the need for students to write short answer responses to any of the questions, and instead necessitates that students circle a response from a set of options (i.e., “yes” or “no”). Table 2 illustrates the questions asked on both the pre- and post-program versions of the survey.

Table 2 Science Interest Survey questions (same pre- and post-program)
Full size table

Draw A Scientist Test (DAST)

There is extensive evidence from the K-12 science educational literature showing that even from a young age, students absorb imagery and influences from popular media and culture depicting what a “typical scientist” should look like and can demonstrate this conformity to stereotypical norms of scientists through illustrations (Chambers 1983; Miele 2014; Steinke et al. 2007). The “Draw A Scientist Test” (DAST) is a validated tool that has been utilized (and modified in recent years) to quantify the myriad ways in which students understand the roles of scientists in society, and how scientists as individuals intersect with culture (Chambers 1983; Farland-Smith 2012; Reinisch et al. 2017; Steinke et al. 2007; Symington and Spurling 1990). The primary strength of the DAST is that it relies heavily on non-verbal/non-written forms of communication (drawings), making it optimal for use with pre-literate children (i.e., grades K-2) (Chambers 1983; Steinke et al. 2007). However, this same strength is also a limitation when it comes to empirically defining, or categorizing, the resulting illustrations of scientists, which was the intention of utilizing this instrument with the YSP student cohort studied (Reinisch et al. 2017; Symington and Spurling 1990). Hence, the DAST was designed in such a way that YSP students were asked to draw what “they believe a scientist looks like” (verbal instructions delivered when test was administered), as well as provide a brief written description of their drawing in the form of a sentence or two. In this way, student illustrations could be sorted into a priori categories by the YSP research assistants, with the written descriptions providing additional context for illustrations in which the characteristics of the scientist depicted were not immediately clear upon review from two or more independent research assistants (i.e., if the illustration did not clearly fall into a category of “female scientist,” the accompanying written description could help to clarify the student’s intention for the scientist’s gender). For the purposes of categorization and statistical analysis (further described in the “Results” section), DASTs were excluded if the drawing was not clearly distinguishable and/or no written description accompanied the drawing. Students were deliberately not provided more detailed instructions when completing the DAST to ensure that their drawings were not potentially influenced by an outsider bias (i.e., from their teachers, the assigned YSP teaching assistant, or any other individuals they may encounter when completing the DAST). Figure 1 depicts some examples of students’ responses (mix of pre- and post-program examples) to the DAST.

Fig. 1
figure 1

Draw A Scientist Test (DAST) examples. Sample responses from students’ Draw A Scientist Tests (DASTs). Many students depicted themselves as scientists, with some even explicitly referencing the lack of URM representation in science (for example, the students’ survey in the top right corner includes the phrase “everyone can be a scientist!” in their written description of their drawing). Many students also referenced specific content or themes they learned through their participation in YSP (in completing the post-program DAST), including the response in the bottom left corner, in which the student talks about the net equation for photosynthesis

Full size image

Experimental design and analysis approach

This study followed a mixed methods pre-/post-test experimental design, utilizing two survey instruments to empirically measure change in student self-efficacy and interest in STEM, as well as changes in their perceptions of scientists, over the course of their participation in the 18-week YSP curriculum. All individual participant data were anonymous, effectively mitigating the need for de-identification to protect participant privacy. The data were also aggregated across school sites and grade levels to conduct a whole-program analysis of the YSP’s efficacy in achieving the previously articulated goals of increasing student self-efficacy and interest in science (measured through responses to the Science Interest Survey) and encouraging students to see themselves as potential future scientists (measured through responses to the DAST).

Six YSP research assistants were trained to process and apply a priori categorizations to the resulting data, and inter-reviewer agreement on these categorizations was quantified through calculations of Cohen’s Kappa (with all calculations yielding a k > 0.5). The a priori categories used in processing and analyzing the DAST results were determined through an independent review, conducted by veteran YSP teaching assistants, of previous years’ randomly selected DAST data, examining patterns and common themes in student illustrations and written descriptions. In the process of collecting and quality-checking the Science Interest Survey results, it was discovered that the language and format of the final two questions were unclear to the vast majority of participating students and as a result, yielded poor quality and/or unusable data, hence their exclusion from the final analyses presented in this paper (the language and formatting of these questions have since been adjusted to facilitate more clear responses and allow for quantitative analyses going forward). Though a combination of quantitative and qualitative methods was used to process the survey data, the analysis was primarily quantitative and included performing N − 1 χ2 Tests on both the Science Interest Survey and DAST results to determine whether differences observed in the paired (pre- and post-program) data were statistically significant at the p < 0.05 (α of 0.05) level. While comparing percentages using this approach does present the benefit of accounting for differences in sample size between the pre- and post-program participant cohorts, it is also prone to an increased Type 1 Error rate; hence, Bonferroni Correction was applied to correct for multiple comparisons.

Results

Science interest surveys

N − 1 χ2 Tests were performed using the openly-available statistical program R/R Studio (2020) to determine whether differences in the overall percentage of students’ responses to four of the six Science Interest Survey Questions was statistically significant at the p < 0.05 (α = 0.05) level. The four independent χ2 tests were conducted using Bonferroni-adjusted α levels of 0.0125 per test (0.05/4). As previously detailed, these four questions were selected for quantitative analysis based upon the quality of the data (student responses) yielded. These four questions were also most pertinent toward addressing the study’s specific aim (to measure changes in student attitudes toward and interest in science over the course of their participation in the YSP 18-week curriculum). The results showed a strong (and in the case of the first question, statistically significant) trend indicating that across all school sites and grade levels, students showed elevated interest in science after participating in the YSP. Specifically, student responses to Question #1 (“How much do you know about science?”) suggested greater confidence in their understanding of scientific principles, particularly those explicitly covered in YSP lessons. The original tallies, percentages of the total sample size, and results from the statistical tests are summarized in Table 3 and graphically represented in Fig. 2.

Table 3 Science Interest Survey (quantitative) results
Full size table
Fig. 2
figure 2

Science Interest Survey results. Graphical summary of differences in percentage of categorical responses to select questions from the Science Interest Survey pre- and post-program. Blue bars correspond to pre-program survey results, and red bars correspond to post-program survey results. Error bars are based on the 95% confidence intervals, and statistical significance is indicated by an asterisk(s) (*p < 0.01, **p < 0.001, and ***p < 0.0001). A Results for Question #1 [“What do you know about science? (Circle One): Nothing, A little, A lot.”] from the survey. B Results for Question #2 [“Do you practice science on your own time? (Circle One): Never, A few times, A lot.”] from the survey. C Results for Question #5 [“Would you encourage a friend or someone in your family to do science? (Circle One): Yes, No.”] from the survey. D Results for Question #6 [“Do you want to be a scientist? (Circle One): Yes, No.”] from the survey

Full size image

Draw A Scientist Test (DAST)

As with the Science Interest Survey pre- and post-program results, N − 1 χ2 Tests were performed to determine whether differences in the overall percentage of students whose drawings fell within each of the a priori categories were statistically significant (p < 0.05). The six independent χ2 tests were conducted using Bonferroni-adjusted α levels of 0.0083 per test (0.05/6). Results from these tests showed a general increase in the number of DAST responses depicting female scientists, scientists of color, and students self-identifying as a scientist, as well as more illustrations of famous scientists from history and/or popular culture, after participating in the YSP 18-week curriculum. However, after correcting for multiple comparisons, none of these pre-/post-program differences were statistically significant. Based upon the independent review of previous years’ DAST responses (in the process of finalizing the a priori categories), it became apparent that many students across all participating school sites and grade levels began to see themselves as future scientists, as more DAST illustrations showed students self-identifying as a scientist (and indicating this in writing along with their illustrations). This aligns with the YSP’s long-term goal of encouraging students to pursue further education and training in STEM professions, and while progress toward this goal cannot be directly measured through examination of the DAST responses alone, the trend from these previously collected data provided support for the initial hypothesis that an increased proportion of students would choose to depict themselves as scientists in the post-program DAST responses, as compared to the pre-program responses. The results for each category are summarized in Table 4 and graphically represented in Fig. 3.

Table 4 Draw A Scientist Test (DAST) results
Full size table
Fig. 3
figure 3

Draw A Scientist Test (DAST) results. Graphical summary of differences in percentage of categorical responses to the DAST, pre- and post-program. Blue bars correspond to pre-program DAST results, and red bars correspond to post-program DAST results. Error bars are based on the 95% confidence intervals

Full size image

Discussion

The overall trend observed from the pre- and post-program data collection shows an increased level of enthusiasm for and interest in pursuing further education in STEM, with many students beginning to identify themselves or those in their community as potential scientists. Furthermore, overlapping increases in the DAST categories of self-identification and female scientist were of interest, as the approximate gender distribution across all schools participating in YSP was 50% female and 50% male. Interestingly, many students chose to draw their own YSP Teaching Assistant on their post-program DASTs, and with over 90% of program teaching staff for the 2018–2019 academic year identifying as female students from URM groups, this increase holds promise in terms of students’ interpretation of contemporary gender roles in the context of STEM professions. As previously noted, one teaching assistant had participated in the program as a student during her elementary school education and made qualitative observations of her students that corroborated many of the quantitative findings from the pre- and post-program data analyses. The teaching assistant identified as a female scientist of color, from the same community sampled for this research, spoke with numerous students at Vermont Avenue Elementary School who felt that their ability to succeed in science academically was enhanced through their participation in YSP.

Taken together, these findings add to the existing body of literature demonstrating the value of early science education and intervention programs that target populations currently under-represented within STEM fields. These results also indicate the potential for the YSP (and similar programs around the country/world) to encourage more students to see themselves as future scientists regardless of their SES, race/ethnicity, gender identity, or any other characteristics that have long hindered individuals from seeking out professions in STEM fields. By changing students’ perceptions of science during a critical period of their development (elementary school age or early adolescence), they may be more likely, as evidenced by other long-term studies tracking students throughout phases of their educational careers, to pursue further scientific exploration of their own volition (Fazio and Gallagher 2019; Saw et al. 2018; Shahali et al. 2018). Moreover, these students will theoretically be motivated and have the academic background to potentially benefit their own communities with their advanced science education, thus completing the cycle of increasing representation in STEM professions (in other words, “paying it forward” to their respective communities) (Hughes et al. 2013; Johnson 2011; Moore 2008; Soldner et al. 2012). The community and students who participate in the YSP are just one representative sample of a broader, more diverse URM population within the United States public educational system. The methods by which the pre- and post-program data were collected and analyzed to measure changes in student attitudes toward science are cost-effective, efficient, and empirically sound, making this approach feasible for similar informal science education programs to adopt as a means of quantifying student attitudes toward science more broadly.

The limitations of this study were primarily related to the quality and quantity of data collected from participating students, as well as the ability to track individual student progress over the course of the 18-week YSP curriculum. Because the survey responses were anonymous and the data from each school site, grade level and individual classroom were aggregated into a single data pool, more robust statistical analyses (such as applying a linear model to the data and/or performing a mixed-effects group analysis with covariates included for age, gender, grade level, etc.) were not possible. For this reason, future evaluations of the YSP, and future research on the effect of supplementary STEM education on student self-efficacy and identity, should be performed and include individualized student data for more detailed, longitudinal analyses by grade level, age, gender, and more. In addition, the survey instruments that were utilized for this specific study have undergone numerous improvements, within the YSP and across other academic institutions, to ensure that these tools are valid, reliable, and objective in the ways that they target student attitudes toward science. Thus, future research will incorporate these updated versions of both the science interest survey and DAST, with additional staff training on how to effectively administer and process the responses from these tools.

The results from the present study, as well as the overall approach for collecting and analyzing these data from a large and diverse student population, have future implications on the ways in which STEM topics are introduced in elementary school classroom settings, and more specifically, how and when these topics are introduced. The YSP survey data show an increased interest in and willingness to engage with science after participating in a hands-on, inquiry-based STEM curriculum, in which students are exposed to the practical applications of what they are learning about in each lesson module and interface with scientific materials, equipment, and terminology that they may not otherwise experience outside of this supplementary curriculum. Thus, ensuring that students have these opportunities to make meaningful connections between their academic understanding of science and the real-world impacts of science at an early stage of their education has strong potential to yield more successful outcomes in longer-term retention of students in STEM fields.

Conclusion

Given the complexity of current global crises, as well as the rate at which STEM fields are advancing to address these crises, the need for early exposure to scientific language and engagement in scientific exploration in the classroom is perhaps greater than ever before. To graduate technologically competent students who can contribute to a growing twenty-first-century global economy, disparities in science educational access must be addressed, particularly in communities that currently lack the financial and other resources necessary to obtain this access (Davenport et al. 2020). The YSP and its evaluation approach present a model which can be generalized to other primary and even secondary educational outreach programs, thus contributing to the body of research examining student attitudes toward science and STEM professions. In the context of elementary school education specifically, students are in a period of early psychological development that presents a unique opportunity to shape their perceptions of learning and the challenges or dangers surrounding science, so that they may acquire a more grounded perspective of what being a scientist truly means, and how to achieve their goals of becoming a scientist (Brotman and Moore 2008; Capobianco et al. 2015). By shifting the priorities of science education from the mastery of content to the level of student engagement, instructors allow their students to have more autonomy and control in the learning process (Carey 2000; Kelly 2000). This paradigm shift could ultimately benefit students at all levels of learning, including at the level of higher education, and is an important aspect to consider when designing curriculum and programs in STEM disciplines.