Workforce Education Models for K-12 STEM Education Programs: Reflections on, and Implications for, the NSF ITEST Program
This article proposes a STEM workforce education logic model, tailored to the particular context of the National Science Foundation’s Innovative Technology Experiences for Students and Teachers (ITEST) program. This model aims to help program designers and researchers address challenges particular to designing, implementing, and studying education innovations in the ITEST program, considering ongoing needs and challenges in STEM workforce education in the USA. It is grounded in conceptual frameworks developed previously by teams of ITEST constituents, for their part intended to frame STEM career education, consider how people select and prepare for STEM careers, and reinforce the important distinction between STEM content and STEM career learnings. The authors take a first step in what they hope will be an ongoing discussion and research agenda by test-fitting assumptions of the model to exploratory case studies of recent NSF ITEST projects. Brief implications for future research and other considerations are provided.
KeywordsNSF National Science Foundation STEM Workforce education Logic model ITEST
Over the past decade or so, numerous reports have reflected concern among policymakers, practitioners, and researchers that the USA is falling short in producing a next generation of science, technology, engineering, and math (STEM) talent to replace those who will soon retire (Business Roundtable 2005; Council on Competitiveness 2008; National Academy of Sciences 2007; National Research Council 2008; National Science Board 2012; National Science Foundation 2010; President’s Council of Advisors on Science and Technology 2010). Of particular national interest is the need for a homegrown STEM workforce development pipeline to fill strategic jobs in secure national laboratories, defense agencies, and other organizations that require US citizenship (Casey 2012). These demands are compounded by the fact that it takes more than a decade to produce a worker capable of filling a high-level scientific research and engineering position, and at least four technicians and technologists to support each working scientist. Moreover, apart from the demand for dedicated STEM professionals, it is increasingly apparent that STEM skills are vital to every sector of the modern economy. In responding to these requirements, our nation’s policymakers have challenged US K-12 education systems to help young people develop the STEM capabilities necessary to fulfill these workforce needs in order to keep our future robust, our economy growing, and our nation safe.
During the past 60 years, career development theorists have made substantial advances in terms of seminal research on career development, helping to shape current understandings of how people develop awareness of and interest in careers, as well as how they prepare for those careers. Developmental theorists generally concur that career development proceeds along a continuum of iterative experiences, in which individuals develop, assess, refine, and act on their career interests, knowledge, and skills, and further, that occupational decision making is a process that addresses complex issues of social and psychological development (Crites 1969; Ginzberg 1972; Ginzberg et al. 1951; Havighurst 1953; Holland 1973; Roe 1957; Super 1951). They are also generally in agreement regarding how this development leads to adult career choices.
In light of the complexities and rapid rates of change in the twenty-first century world in which we live, modern career development theorists have contributed to collective understandings by examining the importance of social interactions and cultural context on career development. Social cognitive career theorists (e.g., Lent et al. 1994) further expanded this perspective to recognize the mutually influential relationship between people and their environment.
With roots in social learning theory (McIlveen and Schulthesiss 2012), today’s social constructionists view career as the result of one’s interactions with others in a larger social, political, and historical context (Young and Popadiuk 2012). They propose that an individual’s career is a result of the discourse in which one engages, interactions with one’s communities of practice, and the systems within which one lives. The life design approach theory (Savickas 2005) posits that individuals construct who they are by developing their social identity as they interact with society (Staunton 2015) and that individuals are authors who, when helped to reflect on the patterns and life themes of their own stories, connect with work settings to construct and reconstruct their careers (Savickas 2005, 2011, 2013; Savickas and Hartung 2012).
Developmental contextualists argue that even though humans are products of biological predispositions and psychological constructions, they are embedded in contexts that shape their developmental pathways (Vondracek et al. 1986; Vondracek 1998; Porfeli and Vondracek 2009). This group further notes that twenty-first century career success is related to the ability of individuals to handle the complexity and change of modern career pathways, the balance of family and work, and the redefinition of “career” found in retirement. In a living systems theory of vocational behavior, Vondracek et al. (2014) propose that each individual can be viewed as a living system (a dynamic, self-directing, self-constructing entity) that interacts with a variety of contexts.
In the modern, industrialized world of two-career households and shared family responsibilities, work contexts are viewed more broadly. Richardson (2012) integrates feminist and social justice perspectives into the discourse on social constructionism of careers by drawing attention to four social contexts within which work occurs: (a) market (paid) work; (b) personal care work (caring for self, dependents, significant others, and community); (c) relationships, in both market work and personal care work; and (d) the social inequities inherent in each. Her work describes how people construct their lives through these contexts. She proposes that twenty-first century students should broaden their understanding of “work” to include personal care work and should construct and reconstruct their life stories to achieve a healthy work–life balance.
Exploration is understood to be an important component of self-constructed identity formation and a cornerstone of career development (Flum and Kaplan 2006), associated with qualities that are much needed in today’s rapidly changing world of work: flexibility and tolerance for ambiguity; a sense of agency, self-determination, autonomy, and successful adjustment; and openness to new experiences. This schema posits that exploratory skills must be learned and that education systems should support mastery goals for exploratory orientation. Renninger and Hidi (2002) associate exploration and interests, suggesting that when students engage in content related to their individual interests, they are pursing mastery goals for exploratory orientation and are better able to concentrate and perform difficult tasks (Renninger 2000). Therefore, when school systems provide opportunities for K-12 students to explore and develop interests in STEM content, students are provided with the necessary foundation to pursue higher-level STEM courses and eventually STEM careers.
Considering these concepts—the developmental nature of career development, the importance of the relationship between individuals and their environments in constructing their career futures, and the benefits of exploration—in light of today’s complex world driven by technology, what should the continuum of STEM career development experiences look like for individuals moving through the US education system? More particularly, how might these ideas be translated to advance program implementation and evaluation research in federal grant-funded programs intended to support STEM career development experiences?
Evolution of an ITEST Workforce Education Model
Considering the range of theoretical orientations described earlier, the helix and matrix models are consistent with the principle ideas that (a) STEM career development is an iterative process, and (b) STEM content and career outcomes build upon one another in nonlinear and recursive ways as young people move along diverse pathways toward STEM careers. Examining these ideas through the collective experiences of ITEST grantees (including the five projects examined as Sample Project Cases for this work), the LRC working group has confidence that their models are effective conceptual elaborations of (a) how students progress iteratively through STEM workforce education activities (the helix), and (b) how outcomes for those students might effectively be defined and categorized (the matrix). Both should be useful schemata that program designers can apply during the planning and implementation stages of workforce development projects similar to those supported by the NSF’s ITEST program.
However, these models come up short in terms of utility for clarifying specific theories of action that must be explicated and tested by program research and/or evaluation efforts—at least to the extent that those studies assess not only the timeliness and quality of program activities and outputs but also generate broader understandings relating to how those activities result in desired outcomes. (Note that we invoke this purpose to make the terms “evaluation” and “research” operationally synonymous for the purposes of this discussion; we make no further effort here to define how they might otherwise be the same or different.) The programmatic goal of evaluation for this purpose requires collecting and analyzing data to test specific theories of how STEM content and career development activities interact with typical elements of teacher professional development and partnerships, and with factors from a project’s cultural context. One tool typically applied to this sort of consideration is the logic model.
About Logic Models
Models are abstractions developed to serve specific purposes, such as the planning of an architectural build or development of an airliner using computer-aided design. The purpose not served by the helix and matrix, but crucial to design and evaluation of STEM workforce education programs, is clarifying the theoretical relationships among factors thought to influence the effectiveness of the education pipeline in meeting workforce needs in STEM and STEM cognate career areas. Viewed broadly, any theory of how the current workforce education system “causes the intended or observed outcomes” (Rogers et al. 2000) should help efforts to strengthen the pipeline to be successful—accepting that the word “causes” may be an overstatement, given the state of true impact research in this area.
Logic models may be tabular or presented graphically (Kellogg Foundation 2004; Weiss 1997; Wholey 1987), as visual representations of relationships among factors related to the problem at hand (Renger and Titcomb 2002). To be effective, logic models should illustrate not only these elements but also the linkages among them (McLaughlin and Jordan 1999), explicating a theory of how they interact to effect change (Chen 1990). As a convention, these connections generally act from left to right, illustrating a series of if–then propositions among the elements.
Ultimately, the clarity of program theory that results from use of an effective logic model should help improve both the quality of implementation and the impact of programs like those typically applied to meet STEM workforce education needs (McLaughlin and Jordan 1999). Shared understanding of a program’s theory of action empowers communication, team cohesion, the dissemination of ideas, and the identification of assumptions (either accurate or unfounded). Explicit program theory can help evaluators and managers identify projects, or elements of projects, that are either crucial to attaining outcomes and thus should be retained, or are redundant or have implausible linkages to outcomes and thus should be eliminated. A logic model can also guide determination of performance measures and inform evaluation study design, data collection, and analysis testing hypotheses indicated by if–then linkages, using data measuring the model elements that they connect (Crew and Anderson 2003). Clarity of program theory can also improve the quality and usefulness of findings if, for example, successes or failures are benchmarked against specific expectations rather than politics or stakeholder intentions (Sengupta 2002).
Toward a STEM Workforce Education Theoretical Model
The logic model is consistent with the precursor models described above in several regards. Outcomes for both students and teachers are aligned with the categories established by the LRC working group’s outcome matrix. Further, logic model elements are color coded by the standard established in the helix model graphic, with most outcomes identified in terms of either STEM careers (red type) or STEM content (black type). Elements illustrated in gray type are operationalized in the ITEST solicitation as not particular to any of the categories in the earlier conceptual models.
This proposed theoretical model should accommodate both traditional and postmodern career development theories mentioned earlier in this discussion. In order to determine their “fit” within STEM work environments, students need to understand the dispositions, knowledge, and skills predominant within STEM workplaces and be able to compare and align to those their own dispositions, knowledge, and skills as they develop over time. As one’s view of career is a result of discourse and interactions with a context, schools need to provide opportunities for students to develop the disciplinary outcomes necessary to engage with STEM technicians and professionals and to activate use of those tools in STEM workplaces. If one accepts that career development is a process occurring in stages that can be guided, education systems seeking to develop students’ STEM content understandings and career capacities would benefit most by connecting learning and careers throughout the K-12 continuum. To be purposeful in STEM workforce education across all grades, both formal and informal educators must develop their own dispositions, knowledge, and skills to support STEM workforce learning. The model proposes that effective teaching and learning actions by educators will promote STEM career development of students leading to persistence in STEM learning and entry into STEM careers. It may go without saying, but career entry outcomes are almost certainly distal enough to K-12 ITEST projects that they will not practically be measured within the scope of a single grant award, even as they must be theoretically linked to funded activities and more proximal outcomes.
It must be noted here that, since this model is new and as yet untested by a substantial body of research, we do not pretend to know how individual elements are theoretically directly linked. Theoretical if–then linkages are implicit between the columns here, following the convention of tabular logic models. The establishment of specific connections among particular elements—variables eventually bearing on workforce placement outcomes—is a key purpose of the research agenda framed by NSF for the ITEST program and illustrated in this model. Doing so through the collection and analysis of data will help determine what strategies show the greatest promise, and ultimately measured effect, on particular workforce outcomes for students.
About Workforce Education Outcomes
For the purpose of the proposed STEM workforce education logic model, we define the term “outcome” very specifically, as a persistent change for individuals or groups that will remain after the program (or other innovation) is completed or no longer acting on them. We focus on this definition as opposed to outputs of program activities, or the goals, objectives, aims, purposes, or other desired results that might be defined for a given program in less-specific terms than we propose here. While not proposing universal definitions, we aim for similar specificity in our usage of additional key terms for this discussion, intended to clarify outcome domains useful to development and assessment of workforce readiness (e.g., Council of Chief State School Officers 2013). Dispositions, in the present context, are attitudes and inclinations, or qualities of mind or character. It may be argued that dispositions toward STEM learning are not malleable, that they are fixed and cannot be influenced by learning experiences, but for the purposes of this discussion, dispositions are held to be flexible in school-aged children and thus open to change by programs or other educational innovations. Knowledge is an accumulation of understanding (i.e., awareness or comprehension) within a particular domain or domains—in this instance, STEM content and STEM careers. A skill is the ability to do something, mental or physical, either to act or exercise agency. Actions are skills made real, observable steps that individuals take toward additional STEM education or career pursuits, both in and out of school.
Outcomes for Educators
For the purposes of this model, the term “educators” (replacing the more specific “teachers”) is framed in the broadest possible sense, to include a full range of titles and roles in both formal (K-12 schools) and informal or out-of-school-time education settings, such as science clubs, competitions, or programs like 4-H. Of particular interest here are career or guidance counselors, whose contributions to students’ decisions about courses and postsecondary education options may well play an important role in priming the STEM education pipeline. Again, teacher knowledge and skills implicitly include the two domains of interest—STEM content and STEM careers—but a third outcome area should also be considered for this group, STEM pedagogy. This domain includes the instructional approaches tailored to the unique characteristics of STEM learning and the effective communication of STEM concepts, such as project-based learning, workplace- or laboratory-based learning, and the use of specialized technologies (Means et al. 2008). Dispositions (e.g., STEM instruction self-efficacy) may also be particularly important for teachers, since those who do not value science or math, or do not feel confident teaching STEM subject matter, are not likely to be effective in realizing targeted outcomes for learners.
Outcomes for Students
In the proposed model, outcomes for students are defined within the now familiar structure of dispositions, knowledge, skills, and actions. It is worth reinforcing here that the arrangement of these outcome domains proposes two particular relationships among them. First, the left-to-right, theoretical if–then organization of the model suggests that dispositions may be precursors to knowledge development, which in turn precedes skill development. Second (and perhaps more certainly) is that achievement of some set of dispositions, knowledge, and skills is necessary in order for students to persist with their STEM education and ultimately enter into STEM careers as productive participants in the workforce.
Testing Model Fit with Examples from Recent ITEST Projects
The ITEST program makes particular mention of the link between STEM learning in both the formal in- and out-of-school-time contexts and workforce education. The current program solicitation (NSF 2015) explains that “ITEST projects must contribute to systematic understanding of student outcomes related to STEM workforce awareness, interests, skills, knowledge or readiness” (p. 4). In fact, STEM workforce constitutes one of the three primary categories that form the strategic framework for the NSF Education and Human Resources (EHR) directorate, in which the ITEST program is located. Workforce investments are intended to improve the education and preparation of a STEM workforce that will be ready to capitalize on unprecedented advances in technology and science and to address global, social, and economic challenges yet to be imagined.
In order to better understand how the proposed model might contribute to broader understandings of STEM workforce learning, data from a sample of ITEST projects were analyzed applying a comparative case study approach (Yin 2003). The results highlight ways in which projects may currently support students’ development of dispositions, knowledge, and skills related to STEM content and careers. With such an emphasis on workforce education and links to connect STEM career development learning experiences with workforce experiences that will increase participation in the STEM pipeline, it may be useful to examine the possibility that such learning–workforce connections are substantially overlooked, underdeveloped, or both across projects in the ITEST program, while being perhaps among the most critical factors in addressing the EHR mandate.
Reflecting on both the directorate’s and the program’s mandates in the area of workforce education, we identified sample ITEST projects, their workforce intentions, and the workforce connections fostered as a result of implementation. These case studies, seen through the helix lens, aim to shed some light on the gaps individual projects might need to address in order for the ITEST program to fulfill its objective of expanding and improving STEM workforce pathways, while exploring the role that the proposed STEM workforce development logic model might contribute to such inquiry.
Limitations of Analysis
Consistent with purposes described by the Common Guidelines for Education Research and Development (Institute of Education Sciences, U.S. Department of Education, & the National Science Foundation 2013), the exploratory study described here is not intended to represent the ITEST program as a whole, but instead to shed light on what kinds of workforce connections and related contributions might be occurring within a subset of current and recent projects. It is intended to be an example of the kinds of data and research that might be addressed by future, more rigorous studies. This study was conducted in a post hoc fashion, in a sense mining specific data from contexts not originally designed to yield such outcomes or serve this exploratory purpose. Nevertheless, we believe this effort provides a window into how workforce and career development linkages may be systematically studied within all ITEST projects, but as well as how those linkages currently appear weaker than EHR staff members likely hope.
In an effort to include a range of projects in our sample, we queried the ITEST principal investigator (PI) and evaluator community about workforce education components in current and former projects. First, we contacted colleagues with whom we have personal relationships, and then, we searched the STELAR (2016) database on ITEST projects. We limited our queries to projects that engaged high school students and STEM professionals, reasoning that these two filters would yield projects that contained at least some deliberate workforce education design, as compared to projects that focused on elementary or middle school students or those in a particular subject area. Of a total of 250 STELAR-inventoried ITEST projects, a search through these filters returned 32 matches. Following outreach to the PIs of these projects, six from that group responded to a short online survey designed to inform the case comparisons.
The data from these six projects were added to those from six others that one of the authors is or has been associated with, providing a sample (mixed random and author biased) of twelve projects from which we might discern where traction is being gained by projects along the workforce education dimensions in the models central to this discussion. The authors acknowledge that, because it is based on convenience sampling methods, this first effort likely suggests findings that may illuminate certain trends and patterns within the samples but are not expected be generalizable across all ITEST projects. Thus, it is the nature of such an exploratory study.
Responses to “To what extent did the workforce development activity…”
Engage students in understanding the STEM workforce
0.00 % (0)
27.27 % (3)
36.36 % (4)
36.36 % (4)
Provide information about STEM careers
0.00 % (0)
27.27 % (3)
9.09 % (1)
63.64 % (7)
Provide actual workplace experiences (e.g., shadowing, internships)
45.45 % (5)
0.00 % (0)
9.09 % (1)
45.45 % (5)
Provide meetings or presentations by STEM professionals
9.09 % (1)
18.18 % (2)
9.09 % (1)
63.64 % (7)
Connect the ITEST project work to STEM careers
0.00 % (0)
18.18 % (2)
27.27 % (3)
54.55 % (6)
Provide visits to STEM workplace sites
36.36 % (4)
9.09 % (1)
9.09 % (1)
45.45 % (5)
Projects applying specific interactions with STEM professionals
Type of interaction
Guest speakers at events or workshops
63.64 % (7)
Contributed to project design
45.45 % (5)
Field trips to workplace or site
72.73 % (8)
Webinar or other online event
18.18 % (2)
9.09 % (1)
Part of core project team
36.36 % (4)
No interaction with STEM professionals
9.09 % (1)
36.36 % (4)
While the intent of this instrument was strictly to gain a quick read on how the targeted ITEST projects have integrated workforce education elements and to establish contact with PIs and evaluators for richer data gathering, a cursory scan of the results illustrates the need for program officers and project managers to consider how they might improve workforce education aspects of programming, both in design and in practice.
Developing a Rating Scale
For discussion purposes, we rated each of the twelve sample projects on how they measured up against the six dimensions described in the helix—teacher professional development, partnerships, and cultural context, each in terms of both STEM content and STEM career development (and so by inference, STEM outcomes). Ratings were based on a composite score of survey responses, conversations with project researchers, and analysis of available project materials and data. Purposes for these analyses were to (a) investigate how the dimensions identified in the LRC working group’s helix model align with those in actual projects, in terms of supporting students’ acquisition of STEM content and career outcomes, and (b) begin developing a method useful to examining these dimensions among different projects.
Discussion of Terms
Rating scale definitions of terms, current versus optimal
Current, from existing projects
Professional development, content (PD CON)
STEM subject-matter content delivered to teachers through professional development events, online modules, workshops, or other interactions (e.g., innovative ways to teach ninth-grade biology topics)
Professional development, career (PD CAR)
Training teachers in how to engage their students in career and workforce education activities related to STEM content
Partnerships, content (Part CON)
Training teachers on operational aspects of developing, maintaining, and growing partnerships that connect STEM learning with workforce opportunities
Partnership activities with students that inform the STEM content being delivered in the program
Partnerships, career (Part CAR)
Extent to which partnerships facilitate career information, access, and experiences in the workforce (teacher, school, or ITEST team driven)
Extent to which partnerships facilitate career information, access, and experiences in the workforce (partner driven)
Cultural context, content (Cultural CON)
Extent to which STEM content delivered to teachers (PD) and students reflects norms and practices related to specific cultural contexts (e.g., language, workforce experience, labor practices, higher education experience)
Cultural context, career (Cultural CAR)
Extent to which career-based and workforce education activities (e.g., internships, site visits, etc.) reflect norms and practices related to specific cultural contexts (e.g., language, workforce experience, labor practices, higher education experience)
Blue bars represent content helix dimensions, and red bars represent career dimensions. Notice that the content bars generally rate higher than those illustrating means of career dimensions. This should cause no surprise, as most NSF ITEST projects develop content first, and PIs may be most familiar with the development and delivery of content, especially through activities relating to professional development (highest rating) and partnerships (second highest rating). Both areas of cultural context scored the lowest.
Sample Project Cases
Five cases highlight how these data apply to actual ITEST projects. These are from the project evaluation portfolio of one of the authors and include a mixture of Strategies-type projects (typically 3-year, single-region-focused, proof-of-concept model projects) and Successful Project Expansion and Dissemination (SPrEaD) projects—the latter generally 4- to 5-year, multi-region, proof-of-scaling model projects, often based on a successful Strategies project. Some projects are current, and some have been completed.
Case 1, completed in 2012, involved urban youth in Detroit. A university team partnered with the city’s Office of Homeland Security and Emergency Management and the public school system to provide over 100 high school students with a two-year training program in geographic information systems and information technology (GIS/IT), culminating in summer internships with a city department or contractor involving work on real-world projects through which they practiced their skills. By the end of the project, students reported significant gains in their determination to seek STEM careers or postsecondary STEM learning opportunities on a range of measures, attributed to participation in internships based on specifically developed STEM expert skills (Xie and Reider 2014). Furthermore, their skill-based experiences introduced them to, and engaged them in, countless opportunities to learn soft skills, including workplace behaviors and protocols (e.g., showing up on time or adhering to a dress code), many of which are well understood by more affluent students but which are not necessarily part of the intrinsic skill set of this population. Finally, many internships took place within the local areas and contexts that had personal meaning to the students, which provided personal motivation for participation and completion, underscoring the value that contexts help shape developmental pathways toward careers (Vondracek et al. 1986; Vondracek 1998; Porfeli and Vondracek 2009).
Case 2 is a SPrEaD project which began in 2014 following the Case 1 Strategies project implemented by the same team. In this scaled-up version of the program, the team has partnered with a state-level instructional agency, a state-level mapping organization, and a statewide online course provider. Compared with its predecessor, with students from a single city participating, this model comprises the entire state of Michigan, including rural communities, at multiple levels of participation—the highest being workforce internships, primarily in fields where jobs often go unfilled.
Case 3 engaged middle school students in three regions of Virginia, following them for 3 years into high school as they learned marine engineering and shipbuilding STEM content, workforce mainstays of the region. Shipyards and related facilities partnered in providing field trips and STEM professional visits to workshops. This project did not have internships, and while the students came from three very different types of communities (port city, landlocked rural, and suburban), cultural context and related connections were not especially well identified.
Case 4, another SPrEaD project, provides minority disadvantaged girls with opportunities to learn and engage with constructive technologies to tell personal stories and develop media artifacts about social challenges to which they have personal connections. These experiences have led to degrees of mastery not realized in their in-school learning environments, mastery that reflects intrinsic motivation (Renninger 2000). Now in its ninth year of existence, and with different program variations supported by different funding sources, the core program operates in multiple states and will be developing an independent university center to support the growing enterprise. The core of its implementation design is cultural context; thus, its program scores in this area are very high. Additionally, through experimentation with different technologies, participants explore their identities as technologists and, ultimately, potential careers. Interviews with girls revealed recurring patterns of both identity and career exploration factors identified by Flum and Kaplan (2006). This program engages several partnerships, but they are devoted to student and site recruitment to meet expansion demands (e.g., public libraries, Boys’ and Girls’ Clubs, church organizations) rather than to model or provide workplace experiences. While the program does have connections with the workforce (workshops often include a female STEM professional giving a talk or presentation), there is no placement component.
Finally, Case 5 is a multistate SPrEaD expansion of a successful earlier project, with over 200 teachers trained in the use of scientific probes and models connected to classroom computers and tablets to help students learn the investigative process and explore how data can justify scientific hypotheses. K-12 classrooms in four states participated formally, but many other students and teachers elsewhere participated informally. This program fell short on partnerships and cultural context related to career development and practices, partly due to the wide range of student ages (Grades 3–12) and differing cultural contexts across all sites. The program made a concerted effort to inform teachers and students about the STEM workforce, with a STEM career Web site that other ITEST projects accessed. The connections were passive, however.
Questions to Consider
At this juncture, it would be appropriate to refocus on a few key questions: What is the impact of ITEST on the workforce trajectory of youth? How does the ITEST program help students apply STEM content to STEM workforce education? How can the ITEST program improve the application of STEM content knowledge to career development?
4-Point ratings of helix dimensions of case study projects, sorted by average rating
Mean of all cases
Program evaluation and internal research in these projects have focused primarily on two realms of outcome, probably not coincidentally those most familiar to school-based education researchers and related to the highest rated dimension among the case study data—changes for students in terms of specific content dispositions, knowledge, and skills; and changes for teachers across the same three outcome areas. The findings about the link between these workforce-directed projects and workforce education outcomes could be more robust. As long-time participants and contributors to the ITEST community, we believe it is important to bring this issue to the forefront. Further, we propose that both program evaluation and ITEST project designs can be improved by (a) attending to action outcomes for both educators and students, as desired results of realizing outcomes in the other three matrix domains, and (b) considering STEM career activities and outcomes with equal attention as those targeting STEM content.
To better tell the ITEST story, the authors of this article propose that PIs and evaluators might engage the concepts illustrated by the helix model, the outcomes of the matrix, and the theoretical framework defined by the workforce education logic model, to develop a set of instruments and methodologies to study in greater depth the workforce outcomes and linkages to development activities that should be inherent in ITEST project designs. Doing so will better inform program teams, the NSF, and ultimately the broader field of STEM workforce education. ITEST provides a rich and accessible testbed like no other, in which the STEM learning community might study these connections. We should consider new tools to take advantage of the opportunity to do so.
This work was borne of a perceived need to address a number of key gaps in knowledge and evaluation research on STEM workforce education. The authors hope that the models and issues discussed in this article have implications for the design, development, and evaluation of K-12 STEM workforce education programs. It is our anticipation that, with these models as a guide, project planners and designers can think more broadly about questions they should be asking and will emphasize a balance of content and career education activities in their STEM projects, programs, and courses. Such projects should look beyond content to collect additional types of data, including measures of more distal outcomes. The ITEST community should benefit from synergies among projects and thus draw more complete and informative conclusions about how ITEST contributes to the STEM workforce. Furthermore, the models proposed in this article should make it easier to leverage the research agenda proposed by the seven guiding questions in the solicitation, as the first step in developing an evaluation framework and set of questions—perhaps even some common measures or instrumentation—that could underpin all ITEST projects, encompassing the full range of aspects of STEM career and workforce education.
These steps have illuminated new challenges and gaps in knowledge and data that might lead toward a better understanding of the impact of ITEST. Our work suggests that while substantial volumes of data and research findings exist relating to the educational components most familiar to us (curriculum, content gains, and professional development outcomes), more is needed to identify how ITEST impacts the career development of future STEM professionals. Possible next steps might include conducting a study to investigate and develop specific instruments, research methods, and recommendations that focus on STEM career and workforce education components. At that point, we may achieve a more balanced representation of the program’s influence on student STEM motivation and participation that will inform education programs and workforce domains as young people pursue educational experiences to prepare for STEM careers.
- Business Roundtable (2005) Tapping America’s potential: the education for innovation initiative. Washington, D.C. July 2005. Retrieved May 7, 2007 from https://www.aau.edu/WorkArea/DownloadAsset.aspx?id=6434
- Casey R (2012) STEM education: preparing for the jobs of the future. U.S. Congress Joint Economic Committee, Washington, DCGoogle Scholar
- Chen HT (1990) Theory-driven evaluations. SAGE Publications, Newbury ParkGoogle Scholar
- Council of Chief State School Officers (2013) Knowledge, skills, and dispositions: the innovation lab network state framework for college, career, and citizenship readiness, and implications for state policy. http://www.ccsso.org/Documents/ILN%20Knowledge%20Skills%20and%20Dispositions%20CCR%20Framework%20February%202013.pdf
- Council on Competitiveness (2008) Thrive: the skills imperative. Council on Competitiveness, Washington, DCGoogle Scholar
- Crites JO (1969) Vocational psychology. McGraw-Hill, New YorkGoogle Scholar
- Ginzberg E (1972) Toward a theory of occupational choice: a restatement. Vocat Guid Q 20:169–176Google Scholar
- Ginzberg E, Ginsburg WS, Axelrad S, Herma JL (1951) Occupational choice: an approach to a general theory. Columbia University Press, New YorkGoogle Scholar
- Havighurst RJ (1953) Human development and education. Longmans Green and Co, New YorkGoogle Scholar
- Holland JL (1973) Making vocational choices: a theory of careers. Prentice Hall, Englewood CliffsGoogle Scholar
- Institute of Education Sciences, U.S. Department of Education, & the National Science Foundation (2013) Common guidelines for education research and development. http://www.nsf.gov/pubs/2013/nsf13126/nsf13126.pdf
- Kellogg Foundation (2004) Using logic models to bring together planning, evaluation, and action: logic model development guide. Retrieved March 15, 2015 from http://www.smartgivers.org/uploads/logicmodelguidepdf.pdf
- McIlveen P, Schulthesiss D (eds) (2012) Social constructionism in vocational psychology and career development. Sense Publishers, RotterdamGoogle Scholar
- Means B, Confrey J, House A, Bhanot R (2008) STEM high schools: specialized science technology engineering and mathematics secondary schools in the U.S. Bill and Melinda Gates Foundation, SeattleGoogle Scholar
- National Academy of Sciences (2007) Rising above the gathering storm: energizing and employing America for a brighter economic future. National Academies Press, Washington, DCGoogle Scholar
- National Research Council (2008) Research on future skill demands: a workshop summary. National Academies Press, Washington, DCGoogle Scholar
- National Science Board (2012) Science and engineering indicators 2012. Arlington: National Science Foundation (NSB 12-01). http://www.nsf.gov/statistics/seind12/?org=NSF
- National Science Foundation (2010) Preparing the next generation of STEM innovators: identifying and developing our nation’s human capital. (Publication No. NSB-10-33). Retrieved October 14, 2010 from http://www.nsf.gov/nsb/publications/2010/nsb1033.pdf
- National Science Foundation (2015) Innovative Technology Experiences for Students and Teachers (ITEST) program solicitation. http://www.nsf.gov/pubs/2015/nsf15599/nsf15599.pdf
- Porfeli EJ, Vondracek FW (2009) Career development, work, and occupational success. In: Smith MC, DeFrates-Densch N (eds) Handbook of research on adult learning and development. Routledge, New York, pp 412–436Google Scholar
- President’s Council of Advisors on Science and Technology (2010) Prepare and inspire: K–12 education in science, technology, engineering and math (STEM) for America’s future. Retrieved October 14, 2010 from http://www.whitehouse.gov/ostp/pcast
- Renninger KA (2000) Individual interest and its implications for understanding intrinsic motivation. In: Sansone C, Harackiewicz JM (eds) Intrinsic and extrinsic motivation: the search for optimal motivation and performance. Routledge, New York, pp 375–404Google Scholar
- Rogers P, Hacsi T, Petrosino A, Huebner T (eds) (2000) Program theory in evaluation: challenges and opportunities. New directions for evaluation, vol 2000, no 87. Jossey-Bass Publishers, San FranciscoGoogle Scholar
- Savickas ML (2005) The theory and practice of career construction. In: Brown SD, Lent RW (eds) Career development and counseling: putting theory and research to work. Wiley, Hoboken, pp 42–70Google Scholar
- Savickas ML (2011) Career counseling. American Psychological Association, Washington, DCGoogle Scholar
- Savickas ML, Hartung PJ (2012) My career story: An autobiographical workbook for life-career success. http://www.vocopher.com/CSI/CCI_workbook.pdf
- Staunton T (2015). Mark Savickas and life design: theories every careers adviser should know. https://runninginaforest.wordpress.com/2015/03/08/mark-savickas-and-life-design-theories-every-careers-adviser-should-know/
- STELAR (2016) STELAR projects. Retrieved March 15, 2016 from http://stelar.edc.org/projects
- Super DE (1951) The psychology of careers: an introduction to vocational development. Harper & Row, New YorkGoogle Scholar
- Vondracek FW, Lerner RM, Schulenberg JE (1986) Career development: a life-span developmental approach. Erlbaum, HillsdaleGoogle Scholar
- Weiss CH (1997) Theory-based evaluation: past, present, and future. In: Rog D, Founier D (eds) Progress and future directions in evaluation, vol 1997, no 76, pp 41–55Google Scholar
- Wholey JS (1987) Evaluability assessment: developing program theory, vol 33. Jossey-Bass, San FranciscoGoogle Scholar
- Yin RK (2003) Case study research: design and methods, 3rd edn. SAGE Publications, Thousand OaksGoogle Scholar
- Young R, Popadiuk N (2012) Social constructionist theories in vocational psychology. In: McIlveen P, Schulthesiss DE (eds) Social constructionism in vocational psychology and career development. Sense Publishers, RotterdamGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.