Findings

Background

There exists a disparity between skills of college graduates and those that employers expect from employees (Employer Survey from NACE, 2015). Employers routinely cite the need to “teach” critical thinking, problem solving, and teamwork to their potential employees. Moreover, other countries routinely outscore the USA in the areas of science, technology, engineering, and mathematics (STEM), and students from low-income families and minority status often have exacerbated deficits in these areas (Sacco et al. 2015; Knopf et al., 2015; NDE, 2015).

A similar call to action comes from the American College Testing group, who report that 78% of high school graduates are not ready for entry-level college classes. We are projected to need several million new college graduates in STEM by 2018 (Carnevale et al. 2010). Nebraska certainly reflects this national trend with a projected 48,000 new STEM positions expected to be left unfilled in 2018 (U.S. Department of Commerce, Economics, and Statistics Administration, 2011). Clearly, a need exists for improved STEM education and training across kindergarten through undergraduate degree completion (K-16).

Given the short timeframe for addressing the STEM deficit, programs that engage youth in structured, high-quality after-school programming that excite youth about STEM areas through consistent, mentored activities while also training STEM undergraduates in career readiness skills have the best chance to make immediate impact as well as establish and maintain a competitive pipeline in STEM. President Obama’s 2010 executive action Reform for the Future supports this in the USA (https://www.whitehouse.gov/issues/education/reform) (Reform for the Future 2012). To address these needs, the University of Nebraska at Omaha (UNO) has implemented a program called NE STEM 4U. This program connects undergraduate STEM majors with students from low-income schools in kindergarten through grade 8 (K-8), to provide after-school, dynamic STEM learning activities that have the potential to benefit both student populations.

NE STEM 4U structure

NE STEM 4U is a Nebraska-based out-of-school time program that takes place in the hours immediately after the school day throughout the academic year. The program targets underprivileged, minority, and underperforming students in schools in Omaha wherein the standardized assessment rates in science and mathematics are the lowest in the state. One hundred percent of participants are on free or reduced lunch. The model of the NE STEM 4U program is outreach through teaching, research, and mentorship by undergraduate students from the University of Nebraska at Omaha (UNO). UNO students are selected through a careful application and interview process. Once accepted, they are trained and led by a team of faculty representing the STEM disciplines as well as education.

Undergraduate students selected for the program participate in a range of training sessions (highlighted in Fig. 1). Students receive mentorship by faculty and specialists in designing and writing lesson plans, engaging youth, giving youth a voice, maintaining an informal classroom atmosphere after school, effective leadership, taking command of a classroom, relating to youth of different ages, multicultural awareness, student safety, and how legislative decisions impact funding for education.

Fig. 1
figure 1

Timeline of the NE STEM 4U program for undergraduate and graduate training and preparation initially and continuously throughout the program

Full size image

The educational experience for NE STEM 4U students is further enhanced through participation in discipline-based education research (DBER). A large body of evidence supports the benefits of research in undergraduate education (Gregg-Jolly et al., 2011; Lopatto, 2007; Lopatto, 2005). Similarly, teaching experiences greatly enhance preparation for STEM careers (Gregg-Jolly et al., 2011). NE STEM 4U students add value to their own educational program through a synergy of teaching and research while inspiring a future generation of STEM students with their outreach. This provides immediate as well as long-term impacts on STEM preparation (DuBois et al., 2002).

Out-of-school time infrastructure in Nebraska

In the state of Nebraska, a state-wide agency called Beyond School Bells and a city-wide, non-profit agency called Collective for Youth manage all such programs. Communication among NE STEM 4U faculty leaders, community groups, and state groups on assessment and impact is frequent and ongoing, thereby making it relatively easy to change direction or to make change happen in a rapid manner.

Objectives of NE STEM 4U

There exists a set of core objectives for the NE STEM 4U program. Presented below are the major overarching research questions for the program for both K-8 student participants and undergraduate mentor participants. Figure 2 is a Venn diagram depicting much of this relationship.

Fig. 2
figure 2

Venn diagram of the threefold model (i.e., teaching, research, and mentorship) of the NE STEM 4U program. UG undergraduates, DBER discipline-based education research

Full size image

Objectives for K-8 student participants

We aim for all K-8 students who are performing these experiments and participating in these activities to behave like scientists and feel like scientists! In our program, they ask questions, make observations, collect data, analyze results, and draw conclusions just as professional scientists do. Taken together our overarching research question related to K-8 student participants, with hands-on explorative learning through NE STEM 4U, do the K-8 student participants show an increase in content knowledge? This overarching question is broken down into specific questions and objectives for K-8 student participants below.

K-8 students participate in activities from all STEM areas. For example, our STEM lessons include the following: biology—strawberry DNA extraction where students extract DNA from strawberries and are taught about DNA, who/what shares DNA, the process of isolating the DNA, etc.; chemistry—dry ice lesson where students participate in a series of activities to learn about dry ice, gas properties, phase changes, sublimation, etc.; engineering—making a fruit battery to teach the concept of generating an electrical current using the natural chemicals found in a fruit or making a solar-powered smore out of an old pizza box; mathematics—calculating heart rate, timing races across the room with different conditions, or quantifying and comparing values by taking the mass of single-stuffed verses double-stuffed Oreos to see if they are in-fact double-stuffed. Moreover, we ask all undergraduate student mentors to spend the last few minutes of each activity discussing a famous STEM professional or a specific career in STEM that complements the activities that day.

Specifically, our objectives for K-8 students served by the program include the following:

  1. 1.

    Demonstrate a greater proficiency in STEM fields;

  2. 2.

    Improve critical thinking and problem solving skills;

  3. 3.

    Decrease behavioral problems that develop when students are not involved in structured activities after school;

  4. 4.

    Participate in activities of which they can easily replicate at home with siblings and parents thereby learning through iteration and gaining confidence as they become the “teachers” (extra materials from each experiment are given to students in order to do just this); and

  5. 5.

    Build upon lessons from the regular school day using hands-on experiments, to provide additional perspective.

Each of these objectives has been linked to assessment instruments.

Objectives for undergraduate student mentor participants

Our program is designed to supplement the academic experience for undergraduates seeking careers as research scientists, teachers, non-profit leaders, or as health-care providers. Our overarching research question related to undergraduate mentors is how does a teaching/mentoring intervention impact undergraduate STEM majors? Consequently, our objectives include developing the following in undergraduate STEM majors:

  1. 1.

    Gain proficiency in STEM fields and enhance the education experience;

  2. 2.

    Engage in community service;

  3. 3.

    Expand confidence and self-esteem by improving organizational skills and STEM content knowledge through mentoring;

  4. 4.

    Prepare for careers in STEM fields by developing communication skills and integrity/dependability; and

  5. 5.

    Increase proficiency in research.

Each of these more specific objectives map to directive research questions and have been linked to assessment instruments.

Metrics collected on NE STEM 4U

A large component of this program is the ongoing DBER research (IRB’s: 548-12-EP and 442-13-EP). This requires the collection of data related to both K-8 student participants and undergraduate mentors. Consequently, we have an expansive data collection methodology (summarized briefly in Table 1). Faculty members work together with graduate students and undergraduate students to collect and analyze these data. These data will be compiled and submitted to journals that report DBER results. Ideally, our publications will help support faculty as they implement the NE STEM 4U model.

Table 1 Summary of undergraduate mentor and K-8 student participant assessments. Included in the table is the list of the assessment itself by participant level, as well as the intended goal and how the assessment was administered (therefore informing data collection)
Full size table

Preliminary results of pilot program

While this report serves as a programmatic introduction, our subsequent reports will describe the results representing specific gains by both K-8 student participants and undergraduate students in the NE STEM 4U program. Importantly, this is a short report summary of a program and, therefore, not a comprehensive report of all data collected per the metrics described in Table 1.

To address our first research question of the impact of this intervention on content knowledge by K-8 participants, our preliminary results (2013–2014; n = 297 K-8) indicated an average of twofold gains (range of 1.7–2.5) in STEM content knowledge for youth that participate at least weekly in the after-school program, NE STEM 4U. Moreover, K-8 student participants in NE STEM 4U perform statistically significantly better on long-term retention assessments of the content (control group earned an average of 30 % on the long-term retention test, NE STEM 4U participants earned an average of 45 %; n = 727; p = 0.0026). While content knowledge in STEM areas is measured through pre- and post-tests in this program, we are also interested in understanding how youth approach STEM inquiry and analysis and these data are obtained using the Dimension of Success tool.

Similar to Knopf et al. (2015), we found that a focused program (i.e., on problem solving and critical thinking in STEM) is effective at improving student gains in STEM. We also, unexpectedly, saw a significant increase in the mentor-mentee relationship with the undergraduates serving as role models to the youth and answering questions about what college is like, how to pick a career, and how to pay for college. Moreover, the youth gain confidence in their abilities to share information in STEM areas, teach one another, and lead experiments.

Importantly, the daily exercises are all hands-on, minds-on experiments encouraging inquiry-based learning where there likely is not a “right” answer and students must discuss and justify their observations. For example, mentor—“Do plants and animals share the same DNA?” Student—“well, they both have DNA, but probably different.” Mentor—“What about a mother and child?” Student—“I would think different, I am a boy and she’s a girl.” Mentor—“What about a skin cell and heart cell?” Student—“they do different things, but they are all my DNA?” (said with puzzled look). All great teaching/learning opportunities are found with these interactions between the mentors and students. In this way, K-8 students are behaving like scientists, practicing investigative iteration, and reflecting upon their interpretations. As a result, these students are well on their way to the skill set that will make them effective in their careers.

Preliminary results from the first academic year of implementation (2013–2014; n = 50) for the undergraduate mentors suggest that the NE STEM 4U program is having a very positive impact on our undergraduate mentors. Specifically, using the daily reflections, over 93 % of mentors stated that the mentoring experience was beneficial to their education and that they felt a sense of accomplishment in helping community members. The following quotes are a few open-ended question responses from undergraduate student mentors regarding their experience (names removed):

The kids really loved the CSI theme and I think [Student mentor] and I were able to use that theme to keep the kids interested and engaged the whole period. They really seemed to be having a good time and most of them did really well on the post-test even though the questions were tough!

I like hearing from the after-school administrators and aides that the students are learning a lot and are having a great time. The fact that the students are talking about our experiments after they leave, and are excited about them, makes me feel like I am doing what I am supposed to be.

I liked that I was able to help students one-on-one how to do Punnett Squares. They understood how to do them better with individual attention.

Furthermore, using the 12- and 24-week interviews, which assessed self-perceived organization skills, content knowledge, preparedness, dependability, and engagement using a ranking from 1 to 10 (10 being the best), we uncovered some notable self-reported growth among the mentors. Specifically, after 12 weeks of mentoring, the average self-rank in all of these categories was a 6.5 or greater, while after 24 weeks of mentoring, the average self-rank was an 8 or greater. As reported in detail in Nelson et al. (under review), the cumulative average percent change from student self-reporting after participating in the NE STEM 4U program indicated the greatest gains in organization, preparedness, and engagement skills. However, students reported gains in all categories over the 24-week time period, with significant gains (p ≤ 0.05 for each category) reported for organization, STEM content knowledge, preparedness, and engagement.

Lastly, after completing the program, 55.5 % of UNO student mentors indicate that they will include mentoring in a future career and 18.5 % reported that this experience caused them to change their career trajectory to teaching science. NE STEM 4U aims to build a pipeline of students able to think critically and address questions in STEM fields.

Discussion

The NE STEM 4U program deploys insight from the literature on how to best provide learning opportunities in STEM disciplines for youth during the out-of-school time. Specifically, the program creates an educational pipeline for students through hands-on experiences in problem solving and critical thinking through daily lessons in STEM areas. The most critical part of this program is the strong tie between the university and community partners. Through these relationships and ongoing discussions, we are able to determine what educational interventions are necessary, which are working, and how we can improve the program in real time.

In this programmatic overview, we describe the structure, objectives, and assessments used in the NE STEM 4U program. We hope that this information will be of benefit to those in other metropolitan cities to replicate such a model. The NE STEM 4U program is meant to provide pre-professional training to college students and to engage K-8 youth in genuine STEM experiences through hands-on, active learning in the after-school setting.

Conclusions

Herein, we described a novel out-of-school time program implemented in Nebraska called NE STEM 4U. As a result of a pilot of this program, we observed gains in content knowledge by youth participants. Additionally, as the programmatic model deploys undergraduates as mentors/teachers of the program, we also observed gains by the undergraduate mentors through self-reported gains in content knowledge, preparedness, organizational skills, and engagement. Importantly, this program has a profound impact on the mentors with more than half reporting that they intend to serve as a mentor in some capacity again in the future. Therefore, after analyzing data from the first year, there is ample evidence to support the hypothesis that we are achieving our core objectives for both K-8 student participants and undergraduate student mentors.