Developing student 21st Century skills in selected exemplary inclusive STEM high schools
- 556 Downloads
There is a need to arm students with noncognitive, or 21st Century, skills to prepare them for a more STEM-based job market. As STEM schools are created in a response to this call to action, research is needed to better understand how exemplary STEM schools successfully accomplish this goal. This conversion mixed method study analyzed student work samples and teacher lesson plans from seven exemplary inclusive STEM high schools to better understand at what level teachers at these schools are engaging and developing student 21st Century skills.
We found of the 67 lesson plans collected at the inclusive STEM high schools, 50 included instruction on 21st Century skills. Most of these lesson plans designed instruction for 21st Century skills at an introductory level. Few lesson plans encouraged multiple 21st Century skills and addressed higher levels of those skills. Although there was not a significant difference between levels of 21st Century skills by grade level, there was an overall trend of higher levels of 21st Century skills demonstrated in lesson plans designed for grades 11 and 12. We also found that lesson plans that lasted three or more days had higher levels of 21st Century skills.
These findings suggest that inclusive STEM high schools provide environments that support the development of 21st Century skills. Yet, more can be done in the area of teacher professional development to improve instruction of high levels of 21st Century skills.
KeywordsSTEM schools 21st Century skills Knowledge construction Real-world problem solving Skilled communication Collaboration Technology Self-regulation
21st Century Learning Design
Information and communication technology
Inclusive STEM high school
National Assessment of Educational Progress
Next-generation science standards
Opportunity Structures for Preparation and Inspiration in STEM
Partnership for 21st Century Learning
Programme for International Student Assessment
Science, technology, engineering, and mathematics
Technology and engineering literacy
School-aged students in the USA are underperforming, particularly in science, technology, engineering, and mathematics (STEM) subjects. National Assessment of Educational Progress (U.S. Department of Education, 2015a) scores show that in science, only 34% of 8th graders are performing at or above proficiency and 12th grade students at or above proficient US students drop to 22%. Similarly, mathematics scores show 33% of 8th graders and 22% of 12th graders were at or above proficiency (U.S. Department of Education, 2015a). Additionally, the US mathematics scores for the Programme for International Student Assessment (PISA) for 2015 were lower than the scores for 2009 and 2012 (Organisation for Economic Co-operation and Development; OECD, 2018). US students not only underachieve in mathematics and science, but are also not engaging successfully in engineering and technology. At the secondary level, there are relatively few students in the USA that take engineering (2%) and computer science (5.7%) (National Science Board, 2016). The NAEP technology and engineering literacy (TEL) assessment found that for technology and engineering literacy, only 43% of 8th graders were at or above the proficiency level (U.S. Department of Education, 2015b). This consistent trend of underperformance has focused many national, state, and local efforts to improve student experiences in integrated STEM subjects (cf. President’s Council of Advisors on Science and Technology, 2010; Texas Education Association (n.d.) for school-aged students and beyond.
The efforts for improvement in STEM teaching in K-12 environments have yielded a slight increase in the enrollment of STEM majors recently (National Science Board, 2016). However, roughly half of students who declare a STEM major when entering college either switch majors or drop out of college (National Science Board, 2016). One approach to helping students persist in undergraduate education is a stronger foundation in content knowledge, academic skills, and noncognitive skills (Farrington et al., 2012). Academic skills, including analysis and problem solving skills, allow students to engage with content knowledge at higher levels of cognition. Noncognitive skills, including study skills, time management, and self-management, assist students in optimizing their ability to gain content knowledge and use their academic skills to solve problems. Students who possess these skills have high-quality academic behaviors, characterized by a pursuit of academic goals despite any setbacks (Farrington et al., 2012).
Because academic skills, noncognitive skills, and content knowledge have fluid definitions and may not be directly observable, for the purposes of this study we used 21st Century skills consisting of knowledge construction, real-world problem solving, skilled communication, collaboration, use of information and communication technology for learning, and self-regulation (Partnership for 21st Century Learning, 2016). Graduates who possess 21st Century skills are sought out by employers (National Research Council, 2013). In the environment of rapid advancements in technology and globalization, employees need to be flexible and perpetual learners in order to keep up with new developments (Bybee, 2013; Johnson, Peters-Burton, & Moore, 2016). There is a need to ensure that students who graduate the K-12 system are adept in 21st Century skills so that they can be successful in this new workforce landscape (Bybee, 2013).
Not only do 21st Century skills help students be successful in all areas of formal school, these skills are also necessary for a person to adapt and thrive in an ever changing world (Partnership for 21st Century Learning, 2016). One movement embracing the need for the development of student 21st Century skills is the proliferation of inclusive STEM high schools (ISHSs), schools that serve all students regardless of prior academic achievement (LaForce et al., 2016; Lynch et al., 2018). ISHSs promote student research experiences by using inquiry-based curricular models to scaffold independent learning and encourage personal responsibility (Tofel-Grehl & Callahan, 2014). The goal for ISHSs to facilitate this type of student-centered learning is to build students’ 21st Century skills such as adaptability, communication, problem solving, critical thinking, collaboration, and self-management (Bybee, 2013; Johnson et al., 2016; LaForce et al., 2016). Although there has been some evidence that not all ISHSs are advantageous in offering STEM opportunities (Eisenhart et al., 2015), there is an accumulation of evidence that ISHSs can increase college and career readiness for students from groups who are typically underrepresented in STEM careers (Erdogan & Stuessy, 2015; Means, Wang, Viki, Peters, & Lynch, 2016). As the number of inclusive STEM schools continue to increase across the USA, there is a need to understand the ways these schools successfully engage students in 21st Century skills. The purpose of this paper is to systematically analyze teacher-constructed lessons and student work from seven exemplar ISHSs in order to better understand how teachers are engaging and developing student 21st Century skills.
Specifically, this study looked at the extent to which teachers at these exemplar ISHSs ask students to practice the 21st Century skills and at the level of student performance of the following categories: (a) knowledge construction, (b) real-world problem solving, (c) skilled communication, (d) collaboration, (e) use of information and communication technology (ICT) for learning, and (f) self-regulation (SRI International, n.d.-a; SRI International, n.d.-b). An examination of the lesson plans and student work products at exemplar ISHSs provides insight into effective development of student 21st Century skills in a variety of contexts.
In an attempt to clearly define the skills, content knowledge and literacies that students would need to be successful in their future endeavors, the Partnership for 21st Century Learning (P21; 2016) created a framework that includes (a) life and career skills; (b) learning and innovation skills; (c) information, media, and technology skills; and (d) key subjects (Partnership for 21st Century Learning, 2016). The first three parts of the framework, (a) life and career skills, (b) learning and innovation skills, and (c) information, media, and technology skills, describe proficiencies or literacies students should develop and can be integrated and developed in any academic lesson. The fourth piece, key subjects, suggests 21st Century interdisciplinary themes or content to engage students in authentic study (Partnership for 21st Century Learning, 2016).
The alignment of 21st century learning components with the 21st century learning design rubrics
P21 framework for 21st century learning
4C’s—Learning and innovation skills
Information, media, and technology skills
Life and career skills
Key subjects—3Rs and 21st century themes
Creativity and innovation
Flexibility and adaptability
Critical thinking and problem solving
Initiative and self-direction
Financial, economic, business and entrepreneurial literacy
Social and cross-cultural skills
Productivity and accountability
Leadership and responsibility
21st century learning design rubrics alignment with P21 framework
Use of ICT
Real-world problem solving
Knowledge construction occurs when students create new knowledge themselves rather than reproducing or consuming information (Prettyman, Ward, Jauk, & Awad, 2012; Shear, Novais, Means, Gallagher, & Langworthy, 2010). When students participate in knowledge construction rather than reproduction, they build a deeper understanding of the content. Learning environments that are designed for knowledge construction promote self-regulated and self-directed learners as well as building grit (Carpenter & Pease, 2013).
Although knowledge construction helps students to build deep understandings and skills to be self-directed and resilient learners, many students are unfamiliar with this approach to learning and frequently need scaffolding to take on joint responsibility of learning (Carpenter & Pease, 2013; Peters, 2010). When transitioning to a more student-centered learning environment that supports knowledge construction, the teacher becomes more of a facilitator rather than a lecturer (McCabe & O’Connor, 2014). A student-centered learning environment encourages students to shift from a paradigm of expecting one convergent answer and toward deeper meaning-making when learning (Peters, 2010). Knowledge construction anchors the development of 21st Century skills because students need to be able to have background knowledge in order to perform the skills in an authentic context.
Real-world problem solving
Sometimes called project-based learning (Warin, Talbi, Kolski, & Hoogstoel, 2016), real-world problem solving is characterized by students working to solve problems that have no current solution and where the students can implement their own approach (Shear et al., 2010). When solving a real-world problem, students work to identify the problem, propose a solution for a specific client, test the solution, and share their ideas (Prettyman et al., 2012; Warin et al., 2016). The design aspect of the process encourages students to be creative and learn from failures (Carroll, 2015). When using real-world problem solving, students develop knowledge in a meaningful way (White & Frederiksen, 1998), must regulate their cognition and behavior in a way to reach their goals (Brown, Bransford, Ferrara, & Campione, 1983; Flavell, 1987), and gain experience defending their choices through evidence and effective communication skills (Voss & Post, 1988).
Teachers can develop real-world problem solving skills in their students by modeling inquiry after research actual scientist are involved in, using databases with real-life data, and evaluating evidence from current events (Chinn & Malhortra, 2002). Designing real-world problem scenarios for the classroom provide a framework by which students can engage in 21st Century learning and can help to encourage a more positive attitude towards STEM careers (Williams & Mangan, 2016). Together, knowledge construction and real-world problem solving create the foundation from which students can engage in self-regulation, collaboration, and communication.
Self-regulation is a key 21st Century skill for independent learners. Students who are self-regulated plan their approach to problem solving, monitor their progress, and reflect on their work given feedback (Shear et al., 2010; Zimmerman, 2000). During the self-regulation process, a student motivates himself or herself to control impulses in order to efficiently solve problems (Carpenter & Pease, 2013; English & Kitsantas, 2013). Fortunately, these skills are teachable; however, students need time to accomplish regulatory tasks and guidance for the key processes of reflection and revision (Zimmerman, 2000). Therefore, long-term projects give a more appropriate time frame than short-term projects to hone these regulatory skills.
Students have different levels of self-regulation (English & Kitsantas, 2013) and teachers may need to integrate strategies and ways of monitoring students into lessons (Bell & Pape, 2014; English & Kitsantas, 2013). Incorporating self-regulated learning strategies helps students to stay engaged and deal with any adversity that may come up in the process (Boekaerts, 2016; Peters & Kitsantas, 2010). A tangible way teachers can support student self-regulation is by using Zimmerman’s (1998) four-stage model of self-regulated learning support: modeling, emulation, self-control, and self-regulation (Peters, 2010). First, teachers explicitly model the target learning strategy that the student should acquire, pointing out key processes (modeling). Second, teachers can provide students with verbal or written support for the key processes of the learning strategy while the student attempts to emulate the modeling from the teacher (emulation). Once students can roughly emulate the learning strategy, the teacher can fade support and have the student try to do the learning strategy on their own (self-control). After students attempt it on their own, the teacher provides feedback to the student to help them improve their attempted learning strategy (self-regulation). When a student can successfully perform the learning strategy on their own, they have become self-regulated in that aspect of their learning. Students who have mastered self-regulated learning have the ability to be proactive in knowledge building and in problem solving, which are characteristics that STEM industry employers value.
Collaboration occurs when students take on roles and interact with one another in groups while working to produce a product (Shear et al., 2010). Collaborative interactions include taking on leadership roles, making decisions, building trust, communicating, reflecting, and managing conflicts (Carpenter & Pease, 2013). Students who collaborate solve problems at higher levels than students who work individually because students respond to feedback and questions to create solutions that better fit the problem (Care, Scoular, & Griffin, 2016). Collaboration is an important skill to enhance knowledge building and problem solving. Conversations among peers can support student self-regulated learning through modeling of verbalized thinking.
“Even the most brilliant scientific discovery, if not communicated widely and accurately, is of little value” (McNutt, 2013, p. 13). For the purpose of this paper, skilled communication is defined as types of communication used to present or explain information, not discourse communication. Skilled communicators present their ideas and demonstrate how they use relevant evidence (Shear et al., 2010). An important part of being able to communicate successfully is the ability to connect a product to the needs of a specific audience or client (Warin et al., 2016). In doing so, the students need to take into account both the media they are using and the ideas they are communicating so that it is appropriate for the audience (Claro et al., 2012; van Laar, van Deursen, van Dijk, & de Haan, 2017). Like collaboration, skilled communication is a necessary process to successfully employ knowledge construction and real-world problem solving.
Use of information and communication technology for learning
When students use information and communication technology (ICT) for learning, they are designing, creating, representing, evaluating, or improving a product, not merely demonstrating their knowledge (Koh, Chai, Benjamin, & Hong, 2015). In doing so, they need to choose how and when to use the ICT as well as know how to recognize credible online resources (Shear et al., 2010). The effective use of ICT requires self-regulation in order to use these tools independently and to keep up with technological advances. Given the continuous advancements in technology, it is essential that students know how to manage and communicate information in order to solve problems (Ainley, Fraillon, Schulz, & Gebhardt, 2016).
Conceptual Model of 21st Century Skills
Knowledge construction and real-world problem solving are the keystones of the model and typically represent the two main goals of student-centered lessons. Knowledge construction is the conceptual formation while real-world problem solving represents the process skills that students are expected to develop. Knowledge construction and real-world problem solving feed each other in a circular fashion. Knowledge construction is built through the inquiry process of real-world problem solving. At the same time, real-world problem solving requires new knowledge to be constructed in order to solve the problem at hand. The connection between knowledge construction and real work problem solving is mediated by collaboration and communication.
While communication and collaboration allow a student to work with others to build their conceptual knowledge and work toward a solution to their real-world problem, self-regulation is an internal process that occurs simultaneously. The student’s self-regulation guides the student’s individual connections, reflections, and revisions between knowledge construction and real-world problem solving.
Information and communication technology provides tools for the students to facilitate communication and collaboration as well as other 21st Century skills. ICT helps to simplify and assist the communication and collaboration for groups of students. ICT can help streamline the process of analysis and record keeping as well as facilitating the sharing ideas with others. It allows students to more easily document their progress and express their ideas for later reflection. Although ICT is not directly connected with other elements in the model, the use of ICT allows for the learning process to be more efficient.
To what extent do teacher lesson plans at exemplar ISHSs exhibit 21st Century learning practices as measured by the 21st Century Learning Design Learning Activity and Student Work Rubrics?
Do teacher lesson plans and student work samples from exemplar ISHSs show differences in rubric scores by grade level?
During the analysis of these questions, a third research question emerged regarding the duration of lessons. The question and rationale can be found in the data analysis section.
The 14 critical components of an inclusive STEM high school
STEM high school inventory: 14 critical components (CC)
CC1: college-prep, STEM focused curriculum for all
CC6: college level coursework
CC11: dynamic assessment systems for continuous improvement
CC2: reform instructional strategies and project-based learning
CC7: well-prepared STEM teachers and professionalized teaching staff
CC12: innovative and responsive leadership
CC3: integrated, innovative technology use
CC8: inclusive STEM mission
CC13: positive school community and culture of high expectations for all
CC4: STEM-rich, informal experiences
CC9: flexible and autonomous administration
CC14: agency and choice
CC5: connections with business, industry and the world of work
CC10: supports for underrepresented students
Cross-case analysis of the eight schools found similarities in how the schools addressed two specific critical components: CC1: college-prep, STEM focused curriculum for all and CC2: reform instructional strategies and project-based learning. From these two critical components, curriculum and instruction, four themes emerged: (a) classroom-related STEM opportunities, (b) cross-cutting school level STEM learning opportunities, (c) school-wide design for STEM opportunities, and (d) responsive design (Peters-Burton, House, Han, & Lynch, 2018). The theme of classroom-related STEM opportunities was characterized by the expectation that teachers act as designers of the curriculum and look beyond the typical textbook for resources. While designing the curriculum, teachers took a mastery learning approach and provided students multiple opportunities to master the material. Through the use of collaborative group projects, summative projects, culminating projects, and interdisciplinary studies, the schools demonstrated a cross-cutting school level approach to the STEM learning. School-wide STEM opportunities included a rigorous curriculum, incorporating engineering classes and/or engineering design thinking, emphasizing connections between the curriculum and real-world examples, as well as building strong collaboration between teachers. Finally, these ISHSs had systems such as data-driven decision making and supports for incoming ninth graders built into their schools as a responsive design. In summary, these schools worked to improve students’ 21st Century skill such as collaboration, problem solving, information and media literacy, and self-directed learning (Lynch et al., 2018).
This study was designed as a conversion mixed methods approach (Tashakkori & Teddlie, 2003) in that qualitative data were transformed into quantitative data using established rubrics. Document analysis was used as a tool to identify occasions of evidence within lessons plans and student work products related to the identified 21st Century skills (Krippendorff, 2012). In this conversion approach, the 21st Century skill demonstrated qualitatively in the documents was scored using the rubrics, ergo integrating qualitative and quantitative methods in the analysis.
The eight exemplar ISHSs for this study came from the same quintain as used by the OSPrI project (Lynch et al., 2018). Because this origin project was a cross-case analysis and the IRB did not allow for school to school comparison, the data collected from individual schools was aggregated as one data source. Protocol for inclusion in the OSPrI study was that the school had no academic admission requirements, self-identified as a STEM school, was in operation for grades 9 through 12, and intentionally recruited students typically underrepresented in STEM. For more information on the demographics of the schools and the selection process, see Lynch et al., 2018. Of the eight schools that were in the original OSPrI project, seven provided teacher lesson plans and/or student work samples during the school visit. All schools have given permission to use their actual names. The sample size from each school was inconsistent, therefore, we treated the data set as one combined group that included all seven schools.
Number of student work samples and teacher lesson plans by content matter and grade level
Student work sample
Teacher lesson plan
Each teacher lesson plan was analyzed using the 21st Century Learning Design (21CLD) Learning Activity Rubric and each student work product was analyzed using the 21st Century Learning Design Student Work Rubric (SRI International, n.d.-a; SRI International, n.d.-b). These instruments were found to be valid and reliable for use in high school classrooms, and Shear et al., 2010 reports the details of the development and validation of the rubrics. Although the student work products were related to the teacher lesson plans, they were analyzed independently according to the protocol of the 21CLD rubrics. The 21CLD Activity Rubric and the 21CLD Student Work Rubric were designed by Microsoft Partner’s in Learning with a collaboration between ITL Research and SRI International (SRI International, n.d.-a; SRI International, n.d.-b). These two 21CLD rubrics were the result of a multi-year project synthesizing research-based practices that promote 21st Century skills (Shear et al., 2010). The rubrics, each 44-pages in length, are available online for public use (https://education.microsoft.com/GetTrained/ITL-Research). The 21CLD rubrics assess teacher lesson plans or student work products on six metrics aligned with 21st Century skills: (a) knowledge construction, (b) real-world problem solving, (c) skilled communication, (d) collaboration, (e) use of ICT for learning, and (f) self-regulation (SRI International, n.d.-a; SRI International, n.d.-b). Collaboration, knowledge construction, and use of ICT score ratings range from one to five while real-world problem solving, self-regulation, and skilled communication score ratings range from one to four.
Examples of teacher lesson plans as assessed with the 21CLD learning activity rubric
Example of a 2
Example of a 4
Students were asked to draw conclusions at the end of a lab activity as to how surface area and weight affect frictional force.
Students were asked to investigate different theories on dark matter and use that knowledge to create advertisements, publications and a talk show in an attempt to sway the public to their side of the debate.
Real-world problem Solving
Students were asked to write an interesting and engaging story that explains calculus through everyday experiences.
Students were divided into groups to refurnish/upgrade the teacher’s house on a budget while balancing needs and wants.
After going on a tour of a gross anatomy lab, students were asked to write a reflection on what they learned on the from the tour and how it applies to their physiology and medical terminology class.
Senior students were required to create a portfolio that included volunteer forms, journal entries, artifacts, and a competed paper with references on their internship experience.
In a lesson where students were working to solve and interpret exponential growth and decay problems the students were expected to work together in table groups.
Students were expected to work in pairs to create a shared presentation and poster on drug awareness. Both students must be prepared to answer questions.
Use of ICT
Students are asked to create a facebook/twitter page, story book, instruction manual, rap, skit, or recipe to explain aerobic and anaerobic respiration.
Students were asked to create a video on You Tube that demonstrates their use of grammar and vocabulary learned in the second semester of Spanish.
Students were asked to graphically represent research on a country and use plotted points to draw the country’s name and flag. Due dates for this project stretched out over two weeks.
Students were asked to create or redesign an invention that help a person with a disability overcome a specific obstacle of their interest. This six-week project had students, plan, create and improve multiple prototypes of their innovation.
Are there differences in the 21 CLD Learning Activity scores of short-term lessons and long-term lessons?
The 21CLD Learning Activity and the 21CLD Student Work Rubrics required a lesson to be long-term order to assess self-regulation. The rubric defined long-term as “if students work on it for a substantive period of time” (SRI International, n.d.-a, p. 32). From our reading of the lesson plans, lessons that were scheduled for three or more days met the criterion of a substantive period of time, while lesson that were scheduled for 1 or 2 days did not meet this criterion. For the purposes of this study, we decided to refine the definition of long-term to be a lesson lasting three or more class periods and a short-term lesson lasting less than three class periods. The analyses for all research questions separated lessons into long-term and short-term in order to clarify the category of self-regulation.
The data were checked for normality, skewness, and outliers; only the teacher lesson plans met all assumptions for an ANOVA (comparison of grade levels) and t test (long-term versus short-term). Due to the small number of student work samples collected (see Table 6), the data related to student work did not meet the assumptions needed to run a t test therefore was not included in this analysis.
Overall rubric scores
Average rubric scores for lesson plans
Rubric metric (max score)
(N = 35)
(N = 32)
(N = 67)
Knowledge construction (5)
Real-world problem solving (4)
Skilled communication (4)
Use of ICT for learning (5)
Total 21CLD score (27)
Average rubric scores for student work samples
Rubric metric (max score)
(N = 22)
(N = 7)
(N = 29)
Knowledge construction (5)
Real-world problem solving (4)
Skilled communication (4)
Use of ICT for learning (5)
Total 21CLD score (27)
21st Century learning by grade
Long-term versus short-term assignments
Independent samples test comparing short-term to long-term lesson plans
99% Confidence interval of the difference
Real-world problem solvinga
Use of ICT for learninga
Total 21CLD scorea
21st Century skills
Overall, the teacher lesson plans collected at the ISHSs showed evidence of addressing 21st Century skills. Nearly 75% of the lessons included at least one 21st Century skill with 67% addressing two or more. Although the majority of lessons addressed multiple 21st Century skills, the rubric scores for these lessons were low because they addressed these skills at a minimal level. For example, a minimal level of collaboration would be instructions to form a group. A high level of collaboration would include defining roles, explicit instructions on how to share responsibility, and evidence of interdependence. Only five lessons showed evidence of multiple 21st Century skills implemented at the highest level, as measured by the 21CLD Learning Activity Rubric.
While assessing the lesson plans, we noted that more explicit instructions in the teacher lesson plans would have resulted in higher rubric scores. Placing students in groups, structuring peer feedback, and having students design a final project for a particular audience are three small changes not seen frequently in the lesson plans that are articulated in the Lesson Plan rubrics to encourage multiple 21st Century skills. When students work in groups, they improve their collaboration and communication skills while constructing knowledge and solving problems (Care et al., 2016; Shear et al., 2010). When teachers incorporate peer feedback into their lesson, students engage in collaboration. Peer feedback also gives students the opportunity to revise their work based on feedback, increasing self-regulation (Shear et al., 2010; Zimmerman, 2000). When students design their final project for a specific target audience, rather than simply displaying their knowledge for the teacher, they work on their skilled communication processes (Claro et al., 2012; van Laar et al., 2017; Warin et al., 2016). In summary, placing students in groups, structuring peer feedback, and having students design a final project for a particular audience provides opportunities for students to practice 21st Century skills.
When lessons addressed more than one 21st Century skill, they usually demonstrated the use of collaboration or communication in real-world problem solving and knowledge construction (Care et al., 2016; Carpenter & Pease, 2013). Thirty-three lesson plans in which real-world problem solving or knowledge construction was evident, 31 showed evidence of collaboration or communication. Similarly, 13 of the 18 student work samples showed evidence of collaboration or communication when real-world problem solving or knowledge construction was practiced. The results from the indirect measures of the rubric build support for a conceptual model connecting the components of 21st Century skills (Fig. 1). There was some evidence demonstrating the support that collaboration and communication have for knowledge construction and real-world problem solving.
The findings of this study point to the likelihood of self-regulation being connected to other 21st Century skills. Each time self-regulation was present in a teacher lesson plan, there was evidence of at least one other 21st Century skill in that lesson. Seventeen of the 23 lesson plans addressing self-regulation included at least three other 21st Century skills, showing evidence that self-regulation is a skill that is related to knowledge construction and real-world problem solving. Our findings reflect the findings of other researchers, in that self-regulation guides the students’ individual connections, reflections, and revisions between knowledge construction and real-world problem solving (Brown et al., 1983; Carpenter & Pease, 2013; Flavell, 1987; Shear et al., 2010).
Evidence from the lessons showed that there was no consistent connection to the use of ICT and the presence of the other 21st Century skills. ICT was seen in both low-scoring lessons as the sole 21st Century skill, as well as in high-scoring lessons in tandem with multiple other 21st Century skills. As in our model, technology is a tool to help facilitate but is not necessary in the development of the other 21st Century skills (Koh et al., 2015; Shear et al., 2010). After examining the data, our model remained unchanged for all 21st Century skills and their relationship to each other.
Grade level differences
Overall, there were no statistically significant differences in the total 21CLD scores across grade levels. This is consistent with the missions of the ISHSs in this study to shift responsibility for learning to the students by weaving 21st Century skills throughout high school grade levels (Lynch et al., 2017). When looking at trends in long-term projects, there was a jump in total 21CLD score for 12th grade. Again, this aligns with the participating schools’ goals of creating an environment where students have a more independent learning experience during their senior year internships, college classes, and specialized programs CC1 (Lynch et al., 2018). This is consistent with the goal of many of the schools to have the students work independently during their senior year either by taking college classes, completing an internship, or taking a career specific set of classes.
Short-term vs. long-term lessons
The data showed that long-term lesson planning had significantly higher scores on the rubric as compared to the short-termed lessons. This difference is consistent with the literature regarding the need for students to have time to develop and practice skills (Lynch et al., 2017; NGSS Lead States, 2013). The extended time allows students to monitor and reflect on their progress while working toward self-regulation of the skill (Carpenter & Pease, 2013; English & Kitsantas, 2013). To truly become self-regulated, students need repeated supported attempts to be able to do it on their own (Zimmerman, 2000).
Although not significant, collaboration was the only rubric metric where the short-term lessons averaged a higher collaboration score than the long-term lessons. Evidence from the lessons show students worked in pairs or groups, but infrequently shared responsibility, made decisions together, or worked interdependently. This leads to the possibility that incorporating the higher levels of collaborations is difficult, even in long-term projects. In addition, evaluating the higher levels of collaboration is difficult to make based solely on documents. Observations would be required to evaluate how the students within the group were interacting with one another.
Because this study used data collected as part of a larger study, there were several limitations. The work collected is a snapshot of the work students were doing at the time of the observation and does not allow for a clear longitudinal look at student growth over time. As stated before, the small student work sample limited what we were able to do with the analysis.
By only analyzing paper copies of the student work, it was not possible to determine a true collaboration score for many of the projects. Higher levels of collaboration such as sharing responsibility, making decisions together, and working interdependently require observation or more detailed notes from the students or teachers. Some lessons may have scored higher in the metric of collaboration had the student interactions been observed or noted.
This study confirmed the presence of all identified 21st Century skills in the lesson plans at the selected exemplar ISHSs serving underrepresented students in STEM: (a) knowledge construction, (b) real-world problem solving, (c) skilled communication, (d) collaboration, (e) use of information and communication technology (ICT) for learning, and (f) self-regulation. In light of the patterns that emerged from the rubrics, we posit that in the lesson plans communication and collaboration are the core 21st Century skills that facilitate knowledge construction and real-world problem solving, while student self-regulation creates efficiencies resulting in improved knowledge construction and real-world problem solving. We also saw in the lesson plans that ICT provides tools to support communication and reflection which leads to knowledge construction and real-world problem solving. To further develop knowledge about how 21st Century skills addressed in lesson plans help to support student work, our model can be a hypothesized starting point to investigate interactions.
While teachers were successful at including 21st Century skills into lessons, very few lessons practiced higher levels of those skills. This could be an indication that high levels of 21st Century skills are difficult to teach explicitly at the high school level. Future studies may investigate why teachers are not frequently incorporating higher level 21st Century skills into their lessons to answer questions as to whether teachers feel that (a) they need more training on incorporating 21st Century skills, (b) students need more practice and scaffolding to build up to higher levels of 21st Century skills, or (c) they need more time for long-term projects to work on the higher level skills.
The use of the 21CLD rubric is a tangible way for teachers to self-assess the level of 21st Century skills in their lessons. Self-evaluation helps encourage reflection, promote professional growth, and recommendations for new aspects of lessons (Akram & Zepeda, 2015; Peterson & Comeaux, 1990). This can also help teachers make the instructions for the development of 21st Century skills more explicit in their lesson. In conducting a self-evaluation, teachers may realize that they do not have a deep understanding of the characteristics of 21st Century skills. If teachers are new to incorporating these skills into their lessons, the teachers may need time to learn the skills themselves before they can incorporate them into their lessons (Yoon et al., 2015). Further studies may examine how teachers use the 21CLD rubric to improve their lesson.
Students need time to grapple with and learn new skills (Lynch et al., 2017; NGSS Lead States, 2013). While we were able to see evidence of higher rubric scores for 21st Century skills for 12th grade students in the lesson plans, due to the convenience sampling of lesson plans and student work samples, we were not able to look at how students’ 21st Century skills were built over time. There is a desire to better understand how ISHSs successfully develop these skills. This includes how schools incorporate and build the 21st Century skills (a) within multiple lessons in one course, (b) across multiple classes over the course of a school year, and (c) throughout the students’ entire high school sequence. Future research may look at a longitudinal study that follows one student’s work over an entire school year to see how the 21CLD scores change. In addition, future studies may also look at how the short-term projects build the skills needed for the students to incorporate higher levels of 21st Century skills in long-term projects.
Publication of this article was funded in part by the George Mason University Libraries Open Access Publishing Fund.
Both authors contributed equally to this manuscript. Both authors read and approved the final manuscript.
This work was conducted by the OSPrI research project, with Sharon Lynch, Tara Behrend, Erin Peters-Burton, and Barbara Means as principal investigators. Funding for OSPrI was provided by the National Science Foundation (DRL 1118851). Any opinions, findings, conclusions, or recommendations are those of the authors and do not necessarily reflect the position or policy of endorsement of the funding agency.
The authors declare that they have no competing interests.
- Akram, M., & Zepeda, S. J. (2015). Development and validation of a teacher self-assessment instrument. Journal of Research and Reflections in Education, 9(2), 134–148.Google Scholar
- Behrend, T. S., Peters-Burton, E. E., Hudson, C., Matray, S., Ford, M., & Lynch, S. J. (2016). STEM High School Inventory. [Measurement instrument]. Retrieved from https://ospri.research.gwu.edu/sites/ospri.research.gwu.edu/files/downloads/CC%20Inventory_FINAL.pdf.
- Boekaerts, M. (2016). Engagement as an inherent aspect of the learning process. Learning and Instruction, 43, 76–83. https://doi.org/10.1016/j.learninstruc.2016.02.001.CrossRefGoogle Scholar
- Brown, A. L., Bransford, J., Ferrara, R., & Campione, J. (1983). Learning, remembering, and understanding. In P. H. Musen (Ed.), Handbook of child psychology (Vol. III, pp. 77–166). New York: Wiley.Google Scholar
- Bybee, R. W. (2013). The case for STEM education. Arlington: NSTA press.Google Scholar
- Carroll, M. (2015). Stretch, dream, and do—A 21st century design thinking & STEM journey. Journal of Research in STEM Education, 1(1), 59–70.Google Scholar
- Claro, M., Preiss, D. D., San Martin, E., Jara, I., Hinostroza, J. E., Valenzuela, S., Cortes, F., & Nussbaun, M. (2012). Assessment of 21st century ICT skills in Chile: Test design and results from high school level students. Computers & Education, 59, 1042–1053. https://doi.org/10.1016/j.compedu.2012.04.004.CrossRefGoogle Scholar
- Eisenhart, M., Weis, L., Allen, C. D., Cipollone, K., Stich, A., & Dominguez, R. (2015). High school opportunities for STEM: Comparing inclusive STEM-focused and comprehensive high schools in two US cities. Journal of Research in Science Teaching, 52(6), 763–789. https://doi.org/10.1002/tea.21213.CrossRefGoogle Scholar
- Erdogan, N., & Stuessy, C. (2015). Examining the role of inclusive STEM schools in the college and career readiness of students in the United States: A multi-group analysis on the outcome of student achievement. Educational Sciences: Theory & Practice, 15(6), 1517–1529. https://doi.org/10.12738/estp.2016.1.0072.CrossRefGoogle Scholar
- Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical literature review. Chicago: University of Chicago Consortium on Chicago School Research.Google Scholar
- Flavell, J. H. (1987). Speculations about the nature and development of metacognition. In F. Weinert & U. R. Kluwe (Eds.), Metacognition, motivation, and understanding (pp. 21–29). Hillsdale: Erlbaum.Google Scholar
- Johnson, C. C., Peters-Burton, E. E., & Moore, T. J. (Eds.). (2016). STEM road map: A framework for integrated STEM education. New York: Routledge.Google Scholar
- Koh, J. H. L., Chai, C. S., Benjamin, W., & Hong, H. Y. (2015). Technological pedagogical content knowledge (TPACK) and design thinking: A framework to support ICT lesson design for 21st century learning. Asia-Pacific Education Researcher (Springer Science & Business Media B.V.), 24(3), 535–543. https://doi.org/10.1007/s40299-015-0237-2.CrossRefGoogle Scholar
- Krippendorff, K. H. (2012). Content analysis: An introduction to its methodology (3rd ed.). Los Angeles: SAGE Publications, Inc..Google Scholar
- Lynch, S. J., House, A., Peters-Burton, E., Behrend, T., Means, B., Ford, M., Spillane, N., Matray, S., Moore, I., Coyne, C., Williams, C., & Corn, J. (2015). A Logic model that describes and explains eight exemplary STEM-focused high schools with diverse student populations. Washington DC: George Washington University OSPrI Project Retrieved from http://ospri.research.gwu.edu.Google Scholar
- Lynch, S. J., Peters-Burton, E. E., Behrend, T., House, A., Ford, M., Spillane, N., Matray, S., Han, E., & Means, B. (2018). Understanding inclusive STEM high schools as opportunity structures for underrepresented students: Critical components. Journal of Research in Science Teaching., 55(5), 712–748. https://doi.org/10.1002/tea.21437.CrossRefGoogle Scholar
- Lynch, S. J., Peters-Burton, E. E., & Ford, M. (2014). Building STEM opportunities for all. Educational Leadership, 72(4), 54–60.Google Scholar
- Lynch, S. J., Spillane, N., House, A., Peters-Burton, E., Behrend, T., Ross, K. M., & Han, E. M. (2017). A policy-relevant instrument case study of an inclusive STEM-focused high school: Manor New Tech High. International Journal of Education in Mathematics, Science and Technology, 5(1), 1–20. https://doi.org/10.18404/ijemst.75656.CrossRefGoogle Scholar
- National Science Board. (2016). Science and engineering indicators 2016. (NSB-2016-1). Arlington: National Science Foundation.Google Scholar
- NGSS Lead States. (2013). Next generation science standards: For states, by states. Washington, DC: The National Academies Press.Google Scholar
- Organisation for Economic Co-operation and Development. (2018). PISA 2015 results in focus. Retrieved from https://www.oecd.org/pisa/pisa-2015-results-in-focus.pdf Google Scholar
- Peters, E. E., & Kitsantas, A. (2010). The effect of nature of science metacognitive prompts on science students’ content and nature of science knowledge, metacognition, and self-regulatory efficacy. School Science and Mathematics, 110, 382–396. https://doi.org/10.1111/j.1949-8594.2010.00050.x.CrossRefGoogle Scholar
- Peters-Burton, E. E., House, A., Han, E., & Lynch, S. (2018). Curriculum and instruction at inclusive STEM high schools. Journal of Research in STEM Education, 4(2), 193–212.Google Scholar
- President’s Council of Advisors on Science and Technology. (2010). Prepare and inspire: K-12 education in science, technology, engineering, and math (STEM) for America’s future. Washington, DC: Executive Office of the President.Google Scholar
- Prettyman, S. S., Ward, C. L., Jauk, D., & Awad, G. (2012). 21st century learners: Voices of students in a one-to-one STEM environment. Journal of Applied Learning Technology, 2(4), 6–15.Google Scholar
- Shear, L., Novais, G., Means, B., Gallagher, L., & Langworthy, M. (2010). ITL research design. Menlo Park: SRI International Retrieved from https://www.sri.com/sites/default/files/publications/itl_research_design_15_nov_2010.pdf.Google Scholar
- SRI International. (n.d.-a). 21CLD learning activity rubrics. Retrieved from https://education.microsoft.com/GetTrained/ITL-Research
- SRI International. (n.d.-b). 21CLD student work rubrics. Retrieved from https://education.microsoft.com/GetTrained/ITL-Research
- Tashakkori, A., & Teddlie, C. (2003). Handbook of mixed methods in social & behavioral research. Thousand Oaks: Sage.Google Scholar
- Texas Education Association (n.d.). Texas science, technology, engineering and mathematics initiative (T-STEM). Retrieved from https://tea.texas.gov/T-STEM/
- U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. (2015a). The nation’s report card: 2015 mathematics and reading assessments. (NCES No. 2015136). Retrieved from https://www.nationsreportcard.gov/reading_math_2015/#?grade=4 Google Scholar
- Voss, J. F., & Post, T. A. (1988). On the solving of ill-structured problems. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. 261–285). Hillsdale: Lawrence Erlbaum.Google Scholar
- Williams, P. J., & Mangan, J. (2016). The effectiveness of using young professionals to influence STEM career choices of secondary school students. Journal of Research in STEM Education, 2(1), 2–18.Google Scholar
- Yoon, S. A., Anderson, E., Koehler-Yom, J., Klopfer, E., Sheldon, J., Wendel, D., Schoenfeld, I., Scheintaub, H., Oztok, M., & Evans, C. (2015). Designing curriculum and instruction for computer-supported complex systems teaching and learning in high school science classrooms. Journal of Research in STEM Education, 1(1), 4–14.Google Scholar
- Zimmerman, B. J. (1998). Developing self-fulfilling cycles of academic regulation: An analysis of exemplary instructional models. In D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulated learning: From teaching to self-reflective practice (pp. 1–19). New York: The Guilford Press.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.